[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:16:40 dist-test-slave-gjrz krb5kdc[23025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:16:40 dist-test-slave-gjrz krb5kdc[23025](info): set up 2 sockets
Aug 11 06:16:40 dist-test-slave-gjrz krb5kdc[23025](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 06:16:42.008335 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.485s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:16:42 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:16:42.168949 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43973
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:43973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:42.629063 23055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:42.629408 23055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:42.629549 23055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:42.636498 23055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:16:42.636593 23055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:42.636672 23055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:42.636744 23055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:16:42.636807 23055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:16:42.646546 23055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:43973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:42.648281 23055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:42.650023 23055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:42.670199 23066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:42.670295 23072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:42.670296 23067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:42.672660 23055 server_base.cc:1034] running on GCE node
I20260811 06:16:42.673491 23055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:42.675390 23055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:42.676648 23055 hybrid_clock.cc:648] HybridClock initialized: now 1786429002676593 us; error 65 us; skew 500 ppm
I20260811 06:16:42.677121 23055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:42 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429002, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:42.683483 23055 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:16:42.685838 23055 webserver.cc:460] Webserver started at http://127.22.121.62:40311/ using document root <none> and password file <none>
I20260811 06:16:42.686852 23055 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:42.686973 23055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:42.687434 23055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:42.691077 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "89b060762fe046c6b4fc2c6fe949ae6b"
format_stamp: "Formatted at 2026-08-11 06:16:42 on dist-test-slave-gjrz"
server_key: "997d2ed45cb04557bd1da33ecbfe7a79"
server_key_iv: "c58b84469f95f5084d1587984d30d34e"
server_key_version: "encryptionkey@0"
I20260811 06:16:42.691847 23055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "89b060762fe046c6b4fc2c6fe949ae6b"
format_stamp: "Formatted at 2026-08-11 06:16:42 on dist-test-slave-gjrz"
server_key: "997d2ed45cb04557bd1da33ecbfe7a79"
server_key_iv: "c58b84469f95f5084d1587984d30d34e"
server_key_version: "encryptionkey@0"
I20260811 06:16:42.700107 23055 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 06:16:42.707468 23086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:42.710347 23055 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260811 06:16:42.710661 23055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "89b060762fe046c6b4fc2c6fe949ae6b"
format_stamp: "Formatted at 2026-08-11 06:16:42 on dist-test-slave-gjrz"
server_key: "997d2ed45cb04557bd1da33ecbfe7a79"
server_key_iv: "c58b84469f95f5084d1587984d30d34e"
server_key_version: "encryptionkey@0"
I20260811 06:16:42.710924 23055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:42.742076 23055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:42.748260 23055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:42.748644 23055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:42.777526 23055 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:43973
I20260811 06:16:42.777627 23178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:43973 every 8 connection(s)
I20260811 06:16:42.779883 23055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:16:42.784808 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23055
I20260811 06:16:42.785300 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:16:42.786814 23179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:42.787766 23012 external_mini_cluster.cc:1428] Setting key b35704fe769a6f7d97378914e1d45053
I20260811 06:16:42.805749 23179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b: Bootstrap starting.
I20260811 06:16:42.812098 23179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:42.814114 23179 log.cc:824] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:42.818599 23179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b: No bootstrap required, opened a new log
Aug 11 06:16:42 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:42.831789 23179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } }
I20260811 06:16:42.832253 23179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:42.832476 23179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b060762fe046c6b4fc2c6fe949ae6b, State: Initialized, Role: FOLLOWER
I20260811 06:16:42.833855 23179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } }
I20260811 06:16:42.834395 23179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:16:42.834805 23179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:16:42.835078 23179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:42.838012 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:42.795455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.22.121.62:43973)
0811 06:16:42.797947 (+ 2492us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:42.797964 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:42.798045 (+ 81us) server_negotiation.cc:407] Connection header received
0811 06:16:42.809250 (+ 11205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:42.809270 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:42.809785 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:42.810169 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:42.814141 (+ 3972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:42.816215 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:42.818235 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:42.819042 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:42.819299 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:42.827224 (+ 7925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:42.827321 (+ 97us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:42.827344 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:42.827390 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:42.831454 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:42.832304 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:42.832313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:42.832321 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:42.832638 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:42.833251 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:42.833256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:42.833261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:42.834277 (+ 1016us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:42.834702 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:42.835633 (+ 931us) server_negotiation.cc:299] Negotiation successful
0811 06:16:42.836089 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2141,"thread_start_us":1842,"threads_started":1}
I20260811 06:16:42.839128 23179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } }
I20260811 06:16:42.840240 23179 leader_election.cc:302] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89b060762fe046c6b4fc2c6fe949ae6b; no voters:
I20260811 06:16:42.841408 23179 leader_election.cc:288] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:16:42.841823 23184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:42.845124 23179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:16:42.845077 23184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [term 1 LEADER]: Becoming Leader. State: Replica: 89b060762fe046c6b4fc2c6fe949ae6b, State: Running, Role: LEADER
I20260811 06:16:42.846130 23184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } }
I20260811 06:16:42.851804 23186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } } }
I20260811 06:16:42.851770 23187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89b060762fe046c6b4fc2c6fe949ae6b. Latest consensus state: current_term: 1 leader_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b060762fe046c6b4fc2c6fe949ae6b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43973 } } }
I20260811 06:16:42.852316 23186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:42.852437 23187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:42.854630 23199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:16:42.864238 23199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:16:42.880937 23199 catalog_manager.cc:1347] Generated new cluster ID: b5aea96ce1534522bc0ebe98c25448e8
I20260811 06:16:42.881039 23199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:16:42.910035 23199 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:16:42.911693 23199 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:16:42.919481 23199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89b060762fe046c6b4fc2c6fe949ae6b: Generated new TSK 0
I20260811 06:16:42.920512 23199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:16:42.974674 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:43.431022 23223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:43.431288 23223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:43.431423 23223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:43.439023 23223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:43.439164 23223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:43.439386 23223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:16:43.448830 23223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:43.449939 23223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:43.451212 23223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:43.466368 23230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:43.466280 23233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:43.466269 23231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:43.467661 23223 server_base.cc:1034] running on GCE node
I20260811 06:16:43.468304 23223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:43.469362 23223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:43.470605 23223 hybrid_clock.cc:648] HybridClock initialized: now 1786429003470508 us; error 39 us; skew 500 ppm
I20260811 06:16:43.470978 23223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:43 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429003, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:43.474227 23223 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:16:43.475767 23223 webserver.cc:460] Webserver started at http://127.22.121.1:32887/ using document root <none> and password file <none>
I20260811 06:16:43.476364 23223 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:43.476482 23223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:43.476764 23223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:43.479542 23223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "2eeba017ec804b23b5ba4b02bc465b27"
format_stamp: "Formatted at 2026-08-11 06:16:43 on dist-test-slave-gjrz"
server_key: "d893b8faac6daa8d9518900604c2feff"
server_key_iv: "fb041c65fbfce594bd981f9572d98a42"
server_key_version: "encryptionkey@0"
I20260811 06:16:43.480283 23223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "2eeba017ec804b23b5ba4b02bc465b27"
format_stamp: "Formatted at 2026-08-11 06:16:43 on dist-test-slave-gjrz"
server_key: "d893b8faac6daa8d9518900604c2feff"
server_key_iv: "fb041c65fbfce594bd981f9572d98a42"
server_key_version: "encryptionkey@0"
I20260811 06:16:43.485355 23223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:16:43.488744 23246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:43.489928 23223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:16:43.490129 23223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "2eeba017ec804b23b5ba4b02bc465b27"
format_stamp: "Formatted at 2026-08-11 06:16:43 on dist-test-slave-gjrz"
server_key: "d893b8faac6daa8d9518900604c2feff"
server_key_iv: "fb041c65fbfce594bd981f9572d98a42"
server_key_version: "encryptionkey@0"
I20260811 06:16:43.490375 23223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:43.511355 23223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:43.515782 23223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:43.516040 23223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:43.517258 23223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:43.518723 23223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:43.518800 23223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:43.518918 23223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:43.518970 23223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:43.560225 23223 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:41005
I20260811 06:16:43.560387 23440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:41005 every 8 connection(s)
I20260811 06:16:43.561419 23223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:16:43.570883 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23223
I20260811 06:16:43.571168 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:16:43.571800 23012 external_mini_cluster.cc:1428] Setting key f2b992d0864780a7bf32ba2c2ee8d4d5
Aug 11 06:16:43 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429003, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:43.586373 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:43.564555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:51167 (local address 127.22.121.62:43973)
0811 06:16:43.565142 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:43.565148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:43.566087 (+ 939us) server_negotiation.cc:407] Connection header received
0811 06:16:43.568211 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:43.568217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:43.568301 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:43.568398 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:43.570123 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:43.571043 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:43.573464 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:43.574227 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:43.574348 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:43.578982 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:43.578990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:43.578995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:43.579034 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:43.582566 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:43.583512 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:43.583516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:43.583519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:43.583685 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:43.584290 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:43.584294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:43.584296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:43.584652 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:43.584764 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:43.585782 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0811 06:16:43.585999 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260811 06:16:43.587997 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:43.565086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:43973 (local address 127.22.121.1:51167)
0811 06:16:43.565923 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:43.565961 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:43.567950 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:43.568687 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:43.568694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:43.569235 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:43.569838 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:43.569857 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:43.571210 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:43.571214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:43.573313 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:43.573327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:43.574542 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:43.574550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:43.574835 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:43.575652 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:43.575680 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:43.578656 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:43.582705 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:43.582713 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:43.582727 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:43.583396 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:43.583789 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:43.583793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:43.583796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:43.584174 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:43.584783 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:43.584791 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:43.585283 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 06:16:43.585673 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 06:16:43.586250 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":409,"threads_started":1}
I20260811 06:16:43.589740 23441 heartbeater.cc:347] Connected to a master server at 127.22.121.62:43973
I20260811 06:16:43.590150 23441 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:16:43.591535 23441 heartbeater.cc:511] Master 127.22.121.62:43973 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
I20260811 06:16:43.595971 23116 ts_manager.cc:194] Registered new tserver with Master: 2eeba017ec804b23b5ba4b02bc465b27 (127.22.121.1:41005)
I20260811 06:16:43.598718 23116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:51167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:16:43.614393 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:44.005312 23451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:44.005585 23451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:44.005760 23451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:44.013365 23451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:44.013481 23451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:44.013633 23451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:16:44.023331 23451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:44.024482 23451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:44.025730 23451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:44.038803 23465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:44.039328 23468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:44.039981 23466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:44.040436 23451 server_base.cc:1034] running on GCE node
I20260811 06:16:44.041095 23451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:44.041962 23451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:44.043159 23451 hybrid_clock.cc:648] HybridClock initialized: now 1786429004043117 us; error 31 us; skew 500 ppm
I20260811 06:16:44.043545 23451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:44.046916 23451 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:16:44.048511 23451 webserver.cc:460] Webserver started at http://127.22.121.2:46051/ using document root <none> and password file <none>
I20260811 06:16:44.049228 23451 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:44.049348 23451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:44.049646 23451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:44.052989 23451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "1a460e14aacc49e4b5514f35666fccc2"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "2160e04b91cec94d047f9db4e0410b7a"
server_key_iv: "523141b847aa8b1cee249c6ed235dea6"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.053723 23451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "1a460e14aacc49e4b5514f35666fccc2"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "2160e04b91cec94d047f9db4e0410b7a"
server_key_iv: "523141b847aa8b1cee249c6ed235dea6"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.059684 23451 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.007s
I20260811 06:16:44.063310 23477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.064548 23451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:16:44.064711 23451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "1a460e14aacc49e4b5514f35666fccc2"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "2160e04b91cec94d047f9db4e0410b7a"
server_key_iv: "523141b847aa8b1cee249c6ed235dea6"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.064879 23451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:44.082450 23451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:44.086655 23451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:44.086912 23451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:44.088056 23451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:44.089409 23451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:44.089486 23451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.089603 23451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:44.089656 23451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.128760 23451 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:36655
I20260811 06:16:44.128866 23684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:36655 every 8 connection(s)
I20260811 06:16:44.130002 23451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:16:44.137576 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23451
I20260811 06:16:44.137796 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:16:44.138355 23012 external_mini_cluster.cc:1428] Setting key 0b4aca61bbe4e3672e55b79eca6b2150
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:44.151096 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.132516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:44163 (local address 127.22.121.62:43973)
0811 06:16:44.133046 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.133053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.133991 (+ 938us) server_negotiation.cc:407] Connection header received
0811 06:16:44.136849 (+ 2858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.136853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.136922 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.137023 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.139088 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.140059 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.141668 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.142113 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.142224 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.145558 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.145563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.145566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.145603 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.148205 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.148860 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.148865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.148869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.149001 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.149400 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.149404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.149406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.149697 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:44.149827 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:44.150517 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 06:16:44.150746 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":241,"threads_started":1}
I20260811 06:16:44.152035 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.132936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:43973 (local address 127.22.121.2:44163)
0811 06:16:44.133812 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:44.133854 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:44.136574 (+ 2720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:44.137315 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:44.137328 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:44.138140 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:44.138915 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:44.138940 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.140217 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.140223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:44.141520 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:44.141533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.142337 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.142346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:44.142578 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.143220 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:44.143242 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:44.145334 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:44.148335 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:44.148344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:44.148359 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:44.148748 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:44.149111 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:44.149116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:44.149120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:44.149294 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:44.149821 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:44.149829 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:44.150176 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 06:16:44.150423 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 06:16:44.150888 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":418,"threads_started":1}
I20260811 06:16:44.153534 23687 heartbeater.cc:347] Connected to a master server at 127.22.121.62:43973
I20260811 06:16:44.153950 23687 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:44.154994 23687 heartbeater.cc:511] Master 127.22.121.62:43973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
I20260811 06:16:44.156625 23116 ts_manager.cc:194] Registered new tserver with Master: 1a460e14aacc49e4b5514f35666fccc2 (127.22.121.2:36655)
I20260811 06:16:44.157644 23116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:44163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:16:44.177738 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:44.594885 23693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:44.595134 23693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:44.595235 23693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 06:16:44.602270 23441 heartbeater.cc:503] Master 127.22.121.62:43973 was elected leader, sending a full tablet report...
W20260811 06:16:44.602365 23693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:44.602475 23693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:44.602617 23693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:16:44.613372 23693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:43973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:44.614524 23693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:44.615788 23693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:44.628886 23702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:44.628926 23703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:44.628961 23705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:44.630394 23693 server_base.cc:1034] running on GCE node
I20260811 06:16:44.630949 23693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:44.631904 23693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:44.633153 23693 hybrid_clock.cc:648] HybridClock initialized: now 1786429004633055 us; error 33 us; skew 500 ppm
I20260811 06:16:44.633438 23693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:44.636495 23693 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:16:44.637836 23693 webserver.cc:460] Webserver started at http://127.22.121.3:40337/ using document root <none> and password file <none>
I20260811 06:16:44.638509 23693 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:44.638639 23693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:44.639048 23693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:44.642349 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "b97de146d17a4b43a3fbbd9fee0e0bc0"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "8c7e4ee3c3f61ba5fe38b5b96adbb7bd"
server_key_iv: "7058ab431773acaa94815fd619da2d9a"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.643081 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "b97de146d17a4b43a3fbbd9fee0e0bc0"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "8c7e4ee3c3f61ba5fe38b5b96adbb7bd"
server_key_iv: "7058ab431773acaa94815fd619da2d9a"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.648420 23693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:16:44.651816 23714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.653065 23693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:16:44.653213 23693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "b97de146d17a4b43a3fbbd9fee0e0bc0"
format_stamp: "Formatted at 2026-08-11 06:16:44 on dist-test-slave-gjrz"
server_key: "8c7e4ee3c3f61ba5fe38b5b96adbb7bd"
server_key_iv: "7058ab431773acaa94815fd619da2d9a"
server_key_version: "encryptionkey@0"
I20260811 06:16:44.653384 23693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:44.680467 23693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:44.684186 23693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:44.684432 23693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:44.685417 23693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:44.686682 23693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:44.686756 23693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.686867 23693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:44.686926 23693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:44.732565 23693 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:43221
I20260811 06:16:44.732630 23922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:43221 every 8 connection(s)
I20260811 06:16:44.733748 23693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:16:44.742401 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23693
I20260811 06:16:44.742585 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:16:44.743037 23012 external_mini_cluster.cc:1428] Setting key a65464c9e9dc318fd4129f9340f19d97
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:44.755296 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.736327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:41073 (local address 127.22.121.62:43973)
0811 06:16:44.736893 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.736897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.737888 (+ 991us) server_negotiation.cc:407] Connection header received
0811 06:16:44.740291 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.740295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.740364 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.740449 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.742212 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.743028 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.745031 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.745588 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.745685 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.749822 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.749827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.749830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.749861 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.752595 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.753300 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.753304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.753306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.753448 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.753811 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.753814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.753816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.754178 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:44.754314 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:44.754818 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 06:16:44.754965 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":350,"threads_started":1}
I20260811 06:16:44.756171 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.736897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:43973 (local address 127.22.121.3:41073)
0811 06:16:44.737740 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:44.737781 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:44.739996 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:44.740679 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:44.740687 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:44.741342 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:44.742046 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:44.742075 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.743192 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.743197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:44.744895 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:44.744910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.745802 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.745810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:44.746336 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.747234 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:44.747255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:44.749620 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:44.752712 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:44.752718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:44.752729 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:44.753206 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:44.753525 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:44.753528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:44.753530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:44.753729 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:44.754267 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:44.754273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:44.754540 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 06:16:44.754745 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 06:16:44.755078 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":374,"threads_started":1}
I20260811 06:16:44.757472 23923 heartbeater.cc:347] Connected to a master server at 127.22.121.62:43973
I20260811 06:16:44.757764 23923 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:44.758813 23923 heartbeater.cc:511] Master 127.22.121.62:43973 requested a full tablet report, sending...
I20260811 06:16:44.760291 23116 ts_manager.cc:194] Registered new tserver with Master: b97de146d17a4b43a3fbbd9fee0e0bc0 (127.22.121.3:43221)
I20260811 06:16:44.761049 23116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:41073
I20260811 06:16:44.771894 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:44.818200 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.803155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.22.121.1:41005)
0811 06:16:44.803838 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.803847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.803880 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:16:44.803943 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.803950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.804167 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.804311 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.805699 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.806722 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.808068 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.808630 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.808755 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.812470 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.812481 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.812495 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.812526 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.815370 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.815991 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.815998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.816001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.816186 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.816557 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.816560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.816562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.816972 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:44.817391 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:44.817603 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:16:44.817882 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":315,"threads_started":1}
W20260811 06:16:44.819528 23296 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45628
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:44.863744 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.849095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45632 (local address 127.22.121.1:41005)
0811 06:16:44.849301 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.849308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.849487 (+ 179us) server_negotiation.cc:407] Connection header received
0811 06:16:44.849687 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.849691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.849786 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.849870 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.851552 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.852411 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.853816 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.854576 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.854692 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.858653 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.858658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.858661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.858694 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.861445 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.862042 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.862045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.862048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.862228 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.862484 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.862487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.862489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.862766 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:44.862972 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:44.863176 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 06:16:44.863383 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:16:44.865547 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23223
I20260811 06:16:44.877287 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.866502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45638 (local address 127.22.121.1:41005)
0811 06:16:44.866648 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.866652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.867009 (+ 357us) server_negotiation.cc:407] Connection header received
0811 06:16:44.867120 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.867123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.867172 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.867244 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.868611 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.869336 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.870637 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.871016 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.871106 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.873084 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.873088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.873091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.873117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.875435 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.875955 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.875959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.875961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.876088 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.876338 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.876341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.876343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.876552 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:44.876717 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:44.876888 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 06:16:44.877022 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:16:44.878015 23435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:44.966051 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23223
I20260811 06:16:44.985806 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23451
Aug 11 06:16:44 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:16:45.001127 23950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:44.986611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41796 (local address 127.22.121.2:36655)
0811 06:16:44.987102 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:44.987110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:44.987136 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:16:44.987198 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:44.987204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:44.987449 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:44.987592 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:44.989066 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.990047 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.991399 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:44.991938 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:44.992062 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:44.995919 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:44.995929 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:44.995942 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:44.995975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:44.998461 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.999068 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.999076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.999078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.999214 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:44.999612 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:44.999617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:44.999619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:44.999943 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:45.000238 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:45.000531 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 06:16:45.000786 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":238,"threads_started":1}
I20260811 06:16:45.002324 23678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:45.082139 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23451
I20260811 06:16:45.102193 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23693
Aug 11 06:16:45 dist-test-slave-gjrz krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:16:45.118045 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:45.103168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.22.121.3:43221)
0811 06:16:45.103833 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:45.103840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:45.103868 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:16:45.103928 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:45.103934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:45.104152 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:45.104300 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:45.105679 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:45.106696 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:45.108125 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:45.108676 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:45.108792 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:45.112658 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:45.112670 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:45.112685 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:45.112727 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:45.115354 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:45.116001 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:45.116008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:45.116010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:45.116148 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:45.116491 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:45.116495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:45.116497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:45.116835 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:45.117184 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:45.117457 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 06:16:45.117716 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":361,"threads_started":1}
I20260811 06:16:45.119295 23913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:45.199947 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23693
I20260811 06:16:45.221390 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23055
I20260811 06:16:45.222622 23173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:45.323163 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23055
2026-08-11T06:16:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4858 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:16:45 dist-test-slave-gjrz krb5kdc[23974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:16:45 dist-test-slave-gjrz krb5kdc[23974](info): set up 2 sockets
Aug 11 06:16:45 dist-test-slave-gjrz krb5kdc[23974](info): commencing operation
krb5kdc: starting...
W20260811 06:16:46.536550 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:16:46 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:16:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:16:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:16:46.692425 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46805
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:46.992022 24003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:46.992293 24003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:46.992427 24003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:46.999320 24003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:16:46.999423 24003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:46.999498 24003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:46.999571 24003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:16:46.999631 24003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:16:47.008788 24003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:47.009971 24003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:47.011238 24003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:47.022221 24010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:47.022253 24011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:47.022423 24015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:47.022993 24003 server_base.cc:1034] running on GCE node
I20260811 06:16:47.023720 24003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:47.024930 24003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:47.026130 24003 hybrid_clock.cc:648] HybridClock initialized: now 1786429007026070 us; error 42 us; skew 500 ppm
I20260811 06:16:47.026417 24003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:47 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429007, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:47.034480 24003 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:16:47.036103 24003 webserver.cc:460] Webserver started at http://127.22.121.62:43549/ using document root <none> and password file <none>
I20260811 06:16:47.036741 24003 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:47.036856 24003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:47.037161 24003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:47.039984 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "27ee36191bb84a5b858b66708faf2deb"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "7c0cc5d5897e4b408a7e9de1f79bbc81"
server_key_iv: "3f2af3da44c963a9c289837c67210860"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.040719 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "27ee36191bb84a5b858b66708faf2deb"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "7c0cc5d5897e4b408a7e9de1f79bbc81"
server_key_iv: "3f2af3da44c963a9c289837c67210860"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.045482 24003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:16:47.048699 24024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:47.049829 24003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 06:16:47.049973 24003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "27ee36191bb84a5b858b66708faf2deb"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "7c0cc5d5897e4b408a7e9de1f79bbc81"
server_key_iv: "3f2af3da44c963a9c289837c67210860"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.050145 24003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:47.064744 24003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:47.068223 24003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:47.068456 24003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:47.085115 24003 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46805
I20260811 06:16:47.085137 24124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46805 every 8 connection(s)
I20260811 06:16:47.086294 24003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:16:47.089630 24125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:47.094435 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24003
I20260811 06:16:47.094647 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:16:47.095238 23012 external_mini_cluster.cc:1428] Setting key 5626efffa354616aa054b7cbddb196ab
I20260811 06:16:47.099758 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb: Bootstrap starting.
I20260811 06:16:47.103085 24125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:47.104215 24125 log.cc:824] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:47.106835 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb: No bootstrap required, opened a new log
Aug 11 06:16:47 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:47.115243 24125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } }
I20260811 06:16:47.115705 24125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:47.115813 24125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ee36191bb84a5b858b66708faf2deb, State: Initialized, Role: FOLLOWER
I20260811 06:16:47.116349 24125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } }
I20260811 06:16:47.116592 24125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:16:47.116714 24125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:16:47.116864 24125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:47.117045 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:47.097075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.22.121.62:46805)
0811 06:16:47.097997 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:47.098008 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:47.098048 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:16:47.099711 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:47.099732 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:47.100115 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:47.100406 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:47.102183 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.103513 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.105124 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.105768 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.105926 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:47.110077 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:47.110084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:47.110101 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:47.110139 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:47.113082 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:47.113816 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:47.113822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:47.113828 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:47.113982 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:47.114328 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:47.114331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:47.114333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:47.114800 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:47.114995 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:47.115567 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 06:16:47.115953 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":458,"threads_started":1}
I20260811 06:16:47.119024 24125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } }
I20260811 06:16:47.119515 24125 leader_election.cc:302] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27ee36191bb84a5b858b66708faf2deb; no voters:
I20260811 06:16:47.120003 24125 leader_election.cc:288] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:16:47.120105 24134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:47.120683 24134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [term 1 LEADER]: Becoming Leader. State: Replica: 27ee36191bb84a5b858b66708faf2deb, State: Running, Role: LEADER
I20260811 06:16:47.121179 24134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } }
I20260811 06:16:47.121781 24125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:16:47.124029 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27ee36191bb84a5b858b66708faf2deb. Latest consensus state: current_term: 1 leader_uuid: "27ee36191bb84a5b858b66708faf2deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } } }
I20260811 06:16:47.124573 24136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27ee36191bb84a5b858b66708faf2deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ee36191bb84a5b858b66708faf2deb" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46805 } } }
I20260811 06:16:47.124754 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:47.124821 24136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:47.128454 24141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:16:47.132251 24141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:16:47.139649 24141 catalog_manager.cc:1347] Generated new cluster ID: cbe2b9d0689e40eab0650778cc3f581d
I20260811 06:16:47.139748 24141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:16:47.154628 24141 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:16:47.156306 24141 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:16:47.164273 24141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27ee36191bb84a5b858b66708faf2deb: Generated new TSK 0
I20260811 06:16:47.164947 24141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:16:47.207885 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:47.517524 24167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:47.517765 24167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:47.517892 24167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:47.524935 24167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:47.525035 24167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:47.525188 24167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:16:47.534430 24167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:47.535564 24167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:47.536784 24167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:47.548799 24183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:47.548836 24184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:47.548877 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:47.549687 24167 server_base.cc:1034] running on GCE node
I20260811 06:16:47.550249 24167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:47.551059 24167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:47.552253 24167 hybrid_clock.cc:648] HybridClock initialized: now 1786429007552199 us; error 48 us; skew 500 ppm
I20260811 06:16:47.552510 24167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:47 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429007, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:47.555611 24167 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:16:47.556936 24167 webserver.cc:460] Webserver started at http://127.22.121.1:42257/ using document root <none> and password file <none>
I20260811 06:16:47.557551 24167 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:47.557672 24167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:47.557951 24167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:47.560788 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "1abd54c4d7aa45c5a3801432d7352dbf"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "0c47d2c573a3b1ab407d4a50c3cd53a2"
server_key_iv: "02137e981383e5bba9235de83e3507e6"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.561537 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "1abd54c4d7aa45c5a3801432d7352dbf"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "0c47d2c573a3b1ab407d4a50c3cd53a2"
server_key_iv: "02137e981383e5bba9235de83e3507e6"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.566362 24167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:16:47.569758 24202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:47.570958 24167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:16:47.571116 24167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "1abd54c4d7aa45c5a3801432d7352dbf"
format_stamp: "Formatted at 2026-08-11 06:16:47 on dist-test-slave-gjrz"
server_key: "0c47d2c573a3b1ab407d4a50c3cd53a2"
server_key_iv: "02137e981383e5bba9235de83e3507e6"
server_key_version: "encryptionkey@0"
I20260811 06:16:47.571362 24167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:47.594594 24167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:47.598603 24167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:47.598834 24167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:47.599746 24167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:47.600978 24167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:47.601051 24167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:47.601146 24167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:47.601189 24167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:47.630452 24167 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45735
I20260811 06:16:47.630498 24415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45735 every 8 connection(s)
I20260811 06:16:47.631680 24167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:16:47.641525 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24167
I20260811 06:16:47.641759 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:16:47.642499 23012 external_mini_cluster.cc:1428] Setting key 266df8ef59899b816a57607ae9e77988
Aug 11 06:16:47 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429007, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:47.650331 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:47.633917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42061 (local address 127.22.121.62:46805)
0811 06:16:47.634302 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:47.634307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:47.635129 (+ 822us) server_negotiation.cc:407] Connection header received
0811 06:16:47.637243 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:47.637250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:47.637335 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:47.637436 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:47.638751 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.639504 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.640901 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.641283 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.641370 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:47.645108 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:47.645113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:47.645117 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:47.645146 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:47.647544 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:47.648205 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:47.648209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:47.648212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:47.648328 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:47.648737 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:47.648742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:47.648744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:47.649006 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:47.649147 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:47.649767 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 06:16:47.649981 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":203,"threads_started":1}
I20260811 06:16:47.651176 24419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:47.634331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46805 (local address 127.22.121.1:42061)
0811 06:16:47.634983 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:47.635018 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:47.636928 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:47.637618 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:47.637625 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:47.638081 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:47.638618 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:47.638635 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.639614 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.639618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:47.640793 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:47.640805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:47.641490 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:47.641498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:47.641779 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:47.642430 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:47.642460 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:47.644901 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:47.647679 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:47.647688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:47.647702 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:47.648094 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:47.648431 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:47.648436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:47.648440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:47.648629 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:47.649135 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:47.649142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:47.649430 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 06:16:47.649664 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 06:16:47.650133 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":256,"threads_started":1}
I20260811 06:16:47.652630 24416 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46805
I20260811 06:16:47.652887 24416 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:47.653582 24416 heartbeater.cc:511] Master 127.22.121.62:46805 requested a full tablet report, sending...
I20260811 06:16:47.655797 24050 ts_manager.cc:194] Registered new tserver with Master: 1abd54c4d7aa45c5a3801432d7352dbf (127.22.121.1:45735)
I20260811 06:16:47.657972 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:42061
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:16:47.680408 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:47.984768 24426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:47.985041 24426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:47.985145 24426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:47.992209 24426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:47.992304 24426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:47.992424 24426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:16:48.001652 24426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:48.002731 24426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:48.003966 24426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:48.016031 24434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:48.016064 24435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:48.016117 24438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:48.016932 24426 server_base.cc:1034] running on GCE node
I20260811 06:16:48.017501 24426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:48.018362 24426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:48.019553 24426 hybrid_clock.cc:648] HybridClock initialized: now 1786429008019499 us; error 43 us; skew 500 ppm
I20260811 06:16:48.019847 24426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:48.022804 24426 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:16:48.024133 24426 webserver.cc:460] Webserver started at http://127.22.121.2:41043/ using document root <none> and password file <none>
I20260811 06:16:48.024778 24426 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:48.024895 24426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:48.025184 24426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:48.028053 24426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "a01052f8d2494370b0bf6ca71eef8db3"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "9213bb6bd9200dc23a1c70d11bf56a52"
server_key_iv: "a89d976468e8e657cf15a082ea196d8a"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.028798 24426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "a01052f8d2494370b0bf6ca71eef8db3"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "9213bb6bd9200dc23a1c70d11bf56a52"
server_key_iv: "a89d976468e8e657cf15a082ea196d8a"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.033604 24426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:16:48.036860 24451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.038060 24426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:16:48.038205 24426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "a01052f8d2494370b0bf6ca71eef8db3"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "9213bb6bd9200dc23a1c70d11bf56a52"
server_key_iv: "a89d976468e8e657cf15a082ea196d8a"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.038376 24426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:48.051973 24426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:48.055866 24426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:48.056107 24426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:48.056938 24426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:48.058141 24426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:48.058216 24426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.058327 24426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:48.058382 24426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.088395 24426 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:43093
I20260811 06:16:48.088474 24657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:43093 every 8 connection(s)
I20260811 06:16:48.089506 24426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:16:48.092459 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24426
I20260811 06:16:48.092625 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:16:48.093034 23012 external_mini_cluster.cc:1428] Setting key b8399141f30a27e810365afb31df4078
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
I20260811 06:16:48.115789 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.091870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:56615 (local address 127.22.121.62:46805)
0811 06:16:48.092028 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.092033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.099904 (+ 7871us) server_negotiation.cc:407] Connection header received
0811 06:16:48.101633 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.101638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.101723 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.101812 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.103623 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.104729 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.106208 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.106812 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.106956 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.110188 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.110194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.110199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.110239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.113082 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.113718 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.113721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.113724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.113843 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.114190 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.114192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.114194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.114484 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.114651 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.115397 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.115575 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:16:48.117079 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.092418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46805 (local address 127.22.121.2:56615)
0811 06:16:48.099759 (+ 7341us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.099799 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.101387 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.102088 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.102099 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.102730 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:48.103448 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.103471 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.104873 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.104877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.106063 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.106075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.107053 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.107058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.107248 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.107920 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:48.107943 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:48.109972 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:48.113219 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:48.113225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:48.113236 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:48.113629 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:48.113931 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:48.113934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:48.113937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:48.114108 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:48.114618 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:48.114624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:48.115000 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.115299 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.115803 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7070,"thread_start_us":6868,"threads_started":1}
I20260811 06:16:48.118397 24658 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46805
I20260811 06:16:48.118677 24658 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:48.119390 24658 heartbeater.cc:511] Master 127.22.121.62:46805 requested a full tablet report, sending...
I20260811 06:16:48.120718 24050 ts_manager.cc:194] Registered new tserver with Master: a01052f8d2494370b0bf6ca71eef8db3 (127.22.121.2:43093)
I20260811 06:16:48.121498 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:56615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:16:48.135056 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:48.439862 24670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:48.440121 24670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:48.440258 24670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:48.447176 24670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:48.447296 24670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:48.447450 24670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:16:48.456547 24670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:48.457647 24670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:48.458853 24670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:48.471203 24680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:48.471303 24679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:48.471248 24684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:48.472082 24670 server_base.cc:1034] running on GCE node
I20260811 06:16:48.472615 24670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:48.473431 24670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:48.474630 24670 hybrid_clock.cc:648] HybridClock initialized: now 1786429008474578 us; error 49 us; skew 500 ppm
I20260811 06:16:48.474908 24670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:48.477763 24670 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:16:48.479056 24670 webserver.cc:460] Webserver started at http://127.22.121.3:34687/ using document root <none> and password file <none>
I20260811 06:16:48.479700 24670 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:48.479815 24670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:48.480098 24670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:48.482795 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "1dd24af53d824f378807be9b3f5950ce"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "79b9f81adcab36877a0ab467c1682b83"
server_key_iv: "1d8cf249e27c90fda7b9c6d284d1e576"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.483580 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "1dd24af53d824f378807be9b3f5950ce"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "79b9f81adcab36877a0ab467c1682b83"
server_key_iv: "1d8cf249e27c90fda7b9c6d284d1e576"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.488241 24670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:16:48.491372 24696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.492542 24670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 06:16:48.492679 24670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1dd24af53d824f378807be9b3f5950ce"
format_stamp: "Formatted at 2026-08-11 06:16:48 on dist-test-slave-gjrz"
server_key: "79b9f81adcab36877a0ab467c1682b83"
server_key_iv: "1d8cf249e27c90fda7b9c6d284d1e576"
server_key_version: "encryptionkey@0"
I20260811 06:16:48.492843 24670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:48.507548 24670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:48.511030 24670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:48.511309 24670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:48.512166 24670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:16:48.513355 24670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:48.513430 24670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.513552 24670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:48.513605 24670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:48.542898 24670 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:44415
I20260811 06:16:48.542953 24908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:44415 every 8 connection(s)
I20260811 06:16:48.544075 24670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:16:48.549016 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24670
I20260811 06:16:48.549214 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:16:48.549734 23012 external_mini_cluster.cc:1428] Setting key 5393d230f6811cad50209e4deb4201a9
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:48.564276 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.546422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:50125 (local address 127.22.121.62:46805)
0811 06:16:48.546571 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.546575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.547678 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 06:16:48.550021 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.550027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.550101 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.550196 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.552050 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.552970 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.554656 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.555085 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.555180 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.558840 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.558844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.558848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.558880 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.561567 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.562188 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.562192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.562194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.562307 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.562627 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.562630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.562632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.562909 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.563027 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.563720 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.563933 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:16:48.565214 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.546811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46805 (local address 127.22.121.3:50125)
0811 06:16:48.547504 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.547545 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.549689 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.550446 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.550454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.551057 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:48.551893 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.551921 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.553153 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.553157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.554520 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.554532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.555295 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.555305 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.555587 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.556262 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:48.556283 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:48.558655 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:48.561690 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:48.561696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:48.561705 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:48.562093 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:48.562380 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:48.562383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:48.562386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:48.562546 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:48.563025 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:48.563032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:48.563386 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.563634 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.564075 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":262,"threads_started":1}
I20260811 06:16:48.566412 24909 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46805
I20260811 06:16:48.566661 24909 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:48.567512 24909 heartbeater.cc:511] Master 127.22.121.62:46805 requested a full tablet report, sending...
I20260811 06:16:48.568774 24050 ts_manager.cc:194] Registered new tserver with Master: 1dd24af53d824f378807be9b3f5950ce (127.22.121.3:44415)
I20260811 06:16:48.569511 24050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:50125
I20260811 06:16:48.576908 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:16:48.597461 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.584132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.22.121.62:46805)
0811 06:16:48.584310 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.584314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.584971 (+ 657us) server_negotiation.cc:407] Connection header received
0811 06:16:48.585097 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.585102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.585159 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.585235 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.587140 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.588074 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.589411 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.589834 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.589937 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.592250 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.592255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.592258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.592287 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.594757 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.595422 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.595428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.595432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.595584 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.595935 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.595940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.595944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.596341 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.596567 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.596972 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.597183 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:16:48.605813 24050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:16:48.614826 24050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:48.660933 24416 heartbeater.cc:503] Master 127.22.121.62:46805 was elected leader, sending a full tablet report...
I20260811 06:16:48.663838 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.630292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.22.121.1:45735)
0811 06:16:48.652722 (+ 22430us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.652731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.653553 (+ 822us) server_negotiation.cc:407] Connection header received
0811 06:16:48.654329 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.654338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.654602 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.654783 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:48.655687 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657598 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.660656 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.662469 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.662607 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.662921 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.663224 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.663408 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22259,"thread_start_us":22120,"threads_started":1}
I20260811 06:16:48.663924 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.629811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45735 (local address 127.0.0.1:34944)
0811 06:16:48.653406 (+ 23595us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.653435 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.654015 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.654786 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.654791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.654815 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:48.655489 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.655503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657699 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.657705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.660461 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.660473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.662811 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.662820 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.663009 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.663048 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.663153 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.663376 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23426,"mutex_wait_us":41,"thread_start_us":23225,"threads_started":1}
I20260811 06:16:48.664201 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.629432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43093 (local address 127.0.0.1:45268)
0811 06:16:48.650821 (+ 21389us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.650870 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.651110 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.654625 (+ 3515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.654631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.654658 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:48.655393 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.655412 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657607 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.657612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.660992 (+ 3380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.661004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.663199 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.663206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.663445 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.663478 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.663557 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.663675 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21138,"mutex_wait_us":140,"thread_start_us":23898,"threads_started":1}
I20260811 06:16:48.664232 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.629930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.22.121.3:44415)
0811 06:16:48.639980 (+ 10050us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.639989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.654364 (+ 14375us) server_negotiation.cc:407] Connection header received
0811 06:16:48.654638 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.654646 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.654856 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.655098 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:48.655907 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657397 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.661269 (+ 3872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.662801 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.662885 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.663160 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.663730 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.663888 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9832,"thread_start_us":9079,"threads_started":1}
I20260811 06:16:48.664909 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.629589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.22.121.2:43093)
0811 06:16:48.653772 (+ 24183us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.653778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.653800 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:16:48.653884 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.653889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.654174 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.654419 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:48.655569 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657490 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.661187 (+ 3697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.662728 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.662865 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.663043 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.664404 (+ 1361us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.664547 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24065,"thread_start_us":23921,"threads_started":1}
I20260811 06:16:48.666167 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.635361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:44415 (local address 127.0.0.1:60168)
0811 06:16:48.654251 (+ 18890us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.654268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.654455 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.655089 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.655093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.655114 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:48.655759 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.655769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.657563 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.657568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.660909 (+ 3341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.660938 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.663357 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.663362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.663546 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.663574 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.665652 (+ 2078us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.665742 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18756,"mutex_wait_us":54,"thread_start_us":15931,"threads_started":1}
I20260811 06:16:48.669031 24290 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3864021cf46c4acb7e31b7de60feb (DEFAULT_TABLE table=test-table [id=0c13b000b3ad4d3cb3cac4f2bc177f5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:48.669260 24786 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3864021cf46c4acb7e31b7de60feb (DEFAULT_TABLE table=test-table [id=0c13b000b3ad4d3cb3cac4f2bc177f5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:48.670634 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3864021cf46c4acb7e31b7de60feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:48.670930 24786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3864021cf46c4acb7e31b7de60feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:48.671653 24549 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3864021cf46c4acb7e31b7de60feb (DEFAULT_TABLE table=test-table [id=0c13b000b3ad4d3cb3cac4f2bc177f5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:48.673036 24549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3864021cf46c4acb7e31b7de60feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:48.683712 24940 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Bootstrap starting.
I20260811 06:16:48.687111 24940 tablet_bootstrap.cc:654] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:48.688174 24941 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Bootstrap starting.
I20260811 06:16:48.688730 24940 log.cc:824] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:48.689623 24943 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Bootstrap starting.
I20260811 06:16:48.692660 24941 tablet_bootstrap.cc:654] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:48.692731 24943 tablet_bootstrap.cc:654] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:48.693683 24943 log.cc:824] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:48.693892 24941 log.cc:824] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:48.694499 24940 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: No bootstrap required, opened a new log
I20260811 06:16:48.694895 24940 ts_tablet_manager.cc:1397] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260811 06:16:48.696465 24943 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: No bootstrap required, opened a new log
I20260811 06:16:48.696774 24943 ts_tablet_manager.cc:1397] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 06:16:48.704228 24943 raft_consensus.cc:348] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.704557 24943 raft_consensus.cc:374] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:48.704667 24943 raft_consensus.cc:729] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01052f8d2494370b0bf6ca71eef8db3, State: Initialized, Role: FOLLOWER
I20260811 06:16:48.705195 24943 consensus_queue.cc:260] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.705582 24941 tablet_bootstrap.cc:492] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: No bootstrap required, opened a new log
I20260811 06:16:48.705924 24941 ts_tablet_manager.cc:1397] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.013s
I20260811 06:16:48.706112 24940 raft_consensus.cc:348] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.706380 24658 heartbeater.cc:503] Master 127.22.121.62:46805 was elected leader, sending a full tablet report...
I20260811 06:16:48.706568 24940 raft_consensus.cc:374] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:48.706663 24940 raft_consensus.cc:729] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abd54c4d7aa45c5a3801432d7352dbf, State: Initialized, Role: FOLLOWER
I20260811 06:16:48.706893 24943 ts_tablet_manager.cc:1428] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 06:16:48.707324 24940 consensus_queue.cc:260] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.708974 24940 ts_tablet_manager.cc:1428] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260811 06:16:48.716164 24941 raft_consensus.cc:348] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.716646 24941 raft_consensus.cc:374] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:48.716775 24941 raft_consensus.cc:729] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd24af53d824f378807be9b3f5950ce, State: Initialized, Role: FOLLOWER
I20260811 06:16:48.717342 24941 consensus_queue.cc:260] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.718637 24909 heartbeater.cc:503] Master 127.22.121.62:46805 was elected leader, sending a full tablet report...
I20260811 06:16:48.719074 24941 ts_tablet_manager.cc:1428] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260811 06:16:48.753392 24952 raft_consensus.cc:482] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:48.753662 24952 raft_consensus.cc:504] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.755688 24952 leader_election.cc:288] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a01052f8d2494370b0bf6ca71eef8db3 (127.22.121.2:43093), 1dd24af53d824f378807be9b3f5950ce (127.22.121.3:44415)
I20260811 06:16:48.764390 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.756099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:41857 (local address 127.22.121.2:43093)
0811 06:16:48.756284 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.756290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.756974 (+ 684us) server_negotiation.cc:407] Connection header received
0811 06:16:48.757123 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.757128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.757199 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.757287 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:48.758066 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.759161 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.762160 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.763368 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.763519 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.763601 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.764016 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.764142 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:16:48.764390 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.756238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:39945 (local address 127.22.121.3:44415)
0811 06:16:48.756403 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.756409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.756937 (+ 528us) server_negotiation.cc:407] Connection header received
0811 06:16:48.757123 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.757128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.757199 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.757287 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:48.758066 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.759483 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.762144 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.763366 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.763523 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.763592 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.764017 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.764145 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:16:48.764679 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.756033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43093 (local address 127.22.121.1:41857)
0811 06:16:48.756886 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.756900 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.757012 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.757317 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.757322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.757367 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:48.757853 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.757871 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.759323 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.759328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.761997 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.762013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.763617 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.763623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.763849 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.763896 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.763987 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.764208 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":231,"spinlock_wait_cycles":35072,"thread_start_us":385,"threads_started":1}
I20260811 06:16:48.764698 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.755890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:44415 (local address 127.22.121.1:39945)
0811 06:16:48.756825 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:48.756845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:48.756994 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:48.757317 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:48.757322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:48.757364 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:48.757948 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.757959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.759614 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.759620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.761998 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:48.762013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.763580 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.763587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:48.763856 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.763903 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:16:48.763987 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 06:16:48.764208 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":227,"spinlock_wait_cycles":14592,"thread_start_us":735,"threads_started":1}
I20260811 06:16:48.765839 24827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3864021cf46c4acb7e31b7de60feb" candidate_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd24af53d824f378807be9b3f5950ce" is_pre_election: true
I20260811 06:16:48.766501 24827 raft_consensus.cc:2454] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1abd54c4d7aa45c5a3801432d7352dbf in term 0.
I20260811 06:16:48.767233 24204 leader_election.cc:302] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abd54c4d7aa45c5a3801432d7352dbf, 1dd24af53d824f378807be9b3f5950ce; no voters:
I20260811 06:16:48.767596 24952 raft_consensus.cc:2781] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:48.767732 24952 raft_consensus.cc:482] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:48.767824 24952 raft_consensus.cc:3037] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:48.767758 24578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3864021cf46c4acb7e31b7de60feb" candidate_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01052f8d2494370b0bf6ca71eef8db3" is_pre_election: true
I20260811 06:16:48.768373 24578 raft_consensus.cc:2454] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1abd54c4d7aa45c5a3801432d7352dbf in term 0.
I20260811 06:16:48.770268 24952 raft_consensus.cc:504] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.770771 24952 leader_election.cc:288] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [CANDIDATE]: Term 1 election: Requested vote from peers a01052f8d2494370b0bf6ca71eef8db3 (127.22.121.2:43093), 1dd24af53d824f378807be9b3f5950ce (127.22.121.3:44415)
I20260811 06:16:48.771178 24578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3864021cf46c4acb7e31b7de60feb" candidate_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01052f8d2494370b0bf6ca71eef8db3"
I20260811 06:16:48.771298 24827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3864021cf46c4acb7e31b7de60feb" candidate_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd24af53d824f378807be9b3f5950ce"
I20260811 06:16:48.771453 24578 raft_consensus.cc:3037] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:48.771486 24827 raft_consensus.cc:3037] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:48.775343 24578 raft_consensus.cc:2454] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1abd54c4d7aa45c5a3801432d7352dbf in term 1.
I20260811 06:16:48.775375 24827 raft_consensus.cc:2454] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1abd54c4d7aa45c5a3801432d7352dbf in term 1.
I20260811 06:16:48.775713 24204 leader_election.cc:302] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abd54c4d7aa45c5a3801432d7352dbf, 1dd24af53d824f378807be9b3f5950ce; no voters:
I20260811 06:16:48.775978 24952 raft_consensus.cc:2781] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:48.776568 24952 raft_consensus.cc:686] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [term 1 LEADER]: Becoming Leader. State: Replica: 1abd54c4d7aa45c5a3801432d7352dbf, State: Running, Role: LEADER
I20260811 06:16:48.776997 24952 consensus_queue.cc:237] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } }
I20260811 06:16:48.782286 24050 catalog_manager.cc:5495] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1abd54c4d7aa45c5a3801432d7352dbf (127.22.121.1). New cstate: current_term: 1 leader_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01052f8d2494370b0bf6ca71eef8db3" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd24af53d824f378807be9b3f5950ce" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 44415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1abd54c4d7aa45c5a3801432d7352dbf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45735 } health_report { overall_health: HEALTHY } } }
W20260811 06:16:48.795656 24913 tablet.cc:2369] T a4b3864021cf46c4acb7e31b7de60feb P 1dd24af53d824f378807be9b3f5950ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:48.821485 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.806568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.22.121.1:45735)
0811 06:16:48.806736 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.806740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.807298 (+ 558us) server_negotiation.cc:407] Connection header received
0811 06:16:48.807427 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.807430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.807489 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.807565 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.808953 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.809646 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.810932 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.811403 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.811494 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.815793 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.815804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.815817 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.815846 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.818791 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.819490 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.819497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.819499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.819760 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.820107 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.820110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.820112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.820415 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.820638 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.820890 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.821047 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 06:16:48.841393 24662 tablet.cc:2369] T a4b3864021cf46c4acb7e31b7de60feb P a01052f8d2494370b0bf6ca71eef8db3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:48.867066 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.853904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.22.121.1:45735)
0811 06:16:48.854127 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.854134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.854468 (+ 334us) server_negotiation.cc:407] Connection header received
0811 06:16:48.854605 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.854609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.854667 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.854750 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.856000 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.856692 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.857838 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.858334 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.858425 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.862353 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.862359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.862362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.862391 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.864882 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.865383 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.865387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.865389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.865506 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.865871 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.865874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.865876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.866155 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.866440 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.866608 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.866769 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 06:16:48.883769 24417 tablet.cc:2369] T a4b3864021cf46c4acb7e31b7de60feb P 1abd54c4d7aa45c5a3801432d7352dbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:16:48 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:48.908622 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.895598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.22.121.1:45735)
0811 06:16:48.895765 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.895771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.896132 (+ 361us) server_negotiation.cc:407] Connection header received
0811 06:16:48.896320 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.896327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.896475 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.896564 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.897824 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.898568 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.899795 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.900204 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.900314 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.903951 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.903956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.903959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.903988 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.906171 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.906756 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.906760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.906763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.906900 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.907241 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.907245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.907247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.907544 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.907779 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.908128 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.908325 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 06:16:48.911003 24248 tablet_service.cc:2908] Rejecting scan request for tablet a4b3864021cf46c4acb7e31b7de60feb: Uninitialized: clean time has not yet been initialized
I20260811 06:16:48.914667 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24167
I20260811 06:16:48.926502 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:48.915492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.22.121.1:45735)
0811 06:16:48.915649 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:48.915655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:48.915973 (+ 318us) server_negotiation.cc:407] Connection header received
0811 06:16:48.916072 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:48.916075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:48.916127 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:48.916196 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:48.917202 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.917848 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.919109 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:48.919541 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:48.919633 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:48.921795 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:48.921800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:48.921803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:48.921830 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:48.924193 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.924872 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.924875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.924877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.924983 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:48.925314 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:48.925317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:48.925318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:48.925525 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:48.925684 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:48.926048 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:16:48.926225 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:16:48.927168 24406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:49.015193 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24167
I20260811 06:16:49.039675 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24426
Aug 11 06:16:49 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:16:49.053093 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:49.040380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.22.121.2:43093)
0811 06:16:49.040537 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:49.040541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:49.040559 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:16:49.040709 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:49.040713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:49.040796 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:49.040877 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:49.042105 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:49.042883 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:49.044225 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:49.044631 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:49.044724 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:49.048295 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:49.048308 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:49.048325 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:49.048359 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:49.050864 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:49.051456 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:49.051463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:49.051466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:49.051605 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:49.051869 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:49.051872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:49.051874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:49.052239 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:49.052449 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:49.052684 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 06:16:49.052841 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:16:49.053717 24646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:49.151733 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24426
I20260811 06:16:49.172796 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24670
Aug 11 06:16:49 dist-test-slave-gjrz krb5kdc[23974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:16:49.186779 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:49.173527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60174 (local address 127.22.121.3:44415)
0811 06:16:49.173679 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:49.173683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:49.173701 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:16:49.173851 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:49.173855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:49.173919 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:49.174011 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:49.175171 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:49.175942 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:49.177206 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:49.177606 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:49.177692 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:49.181811 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:49.181826 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:49.181843 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:49.181876 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:49.184590 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:49.185194 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:49.185200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:49.185202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:49.185335 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:49.185619 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:49.185622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:49.185624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:49.185960 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:49.186169 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:49.186368 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 06:16:49.186524 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:16:49.187412 24903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:49.272233 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24670
I20260811 06:16:49.291738 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24003
I20260811 06:16:49.292781 24117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:49.395826 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24003
2026-08-11T06:16:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4074 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:16:49 dist-test-slave-gjrz krb5kdc[24992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:16:49 dist-test-slave-gjrz krb5kdc[24992](info): set up 2 sockets
Aug 11 06:16:49 dist-test-slave-gjrz krb5kdc[24992](info): commencing operation
krb5kdc: starting...
W20260811 06:16:50.609576 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:16:50 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:16:50.769897 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46701
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:51.076059 25012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:51.076323 25012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:51.076448 25012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:51.083285 25012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:16:51.083387 25012 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:16:51.083448 25012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:51.083531 25012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:51.083600 25012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:16:51.083681 25012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:16:51.092803 25012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46701
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:51.093883 25012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:51.095093 25012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:51.105906 25017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:51.105842 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:51.105885 25018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:51.107112 25012 server_base.cc:1034] running on GCE node
I20260811 06:16:51.107867 25012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:51.109074 25012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:51.110287 25012 hybrid_clock.cc:648] HybridClock initialized: now 1786429011110227 us; error 36 us; skew 500 ppm
I20260811 06:16:51.110579 25012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:51 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429011, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:51.113673 25012 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:16:51.115061 25012 webserver.cc:460] Webserver started at http://127.22.121.62:33493/ using document root <none> and password file <none>
I20260811 06:16:51.115731 25012 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:51.115826 25012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:51.116118 25012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:51.118817 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "c971479a50a14aefa45ef7dfeac4c4a9"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "6b0189f6e3dc9cae0964085da9d479c2"
server_key_iv: "033bfa5e2f2264fdf82bc54a16fcacf5"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.119570 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "c971479a50a14aefa45ef7dfeac4c4a9"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "6b0189f6e3dc9cae0964085da9d479c2"
server_key_iv: "033bfa5e2f2264fdf82bc54a16fcacf5"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.124218 25012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:16:51.127316 25034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:51.128446 25012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:16:51.128600 25012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "c971479a50a14aefa45ef7dfeac4c4a9"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "6b0189f6e3dc9cae0964085da9d479c2"
server_key_iv: "033bfa5e2f2264fdf82bc54a16fcacf5"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.128782 25012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:51.147210 25012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:51.150554 25012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:51.150779 25012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:51.169921 25012 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46701
I20260811 06:16:51.169955 25145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46701 every 8 connection(s)
I20260811 06:16:51.171057 25012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:16:51.172379 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25012
I20260811 06:16:51.172622 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:16:51.173420 23012 external_mini_cluster.cc:1428] Setting key 412ba3dcc9f6b684234e227783fe53e8
I20260811 06:16:51.175500 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:16:51 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:51.187951 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Bootstrap starting.
I20260811 06:16:51.190927 25146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:51.192147 25146 log.cc:824] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:51.194782 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:51.175360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.22.121.62:46701)
0811 06:16:51.176064 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:51.176075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:51.176113 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:16:51.177632 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:51.177652 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:51.178000 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:51.178293 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:51.179975 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.181275 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.182942 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.183602 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.183728 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:51.188308 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:51.188317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:51.188329 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:51.188360 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:51.191020 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.191817 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.191823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.191829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.191988 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.192359 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.192362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.192364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.193013 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:51.193312 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:51.193576 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:16:51.193886 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":333,"threads_started":1}
I20260811 06:16:51.194954 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: No bootstrap required, opened a new log
I20260811 06:16:51.201749 25146 raft_consensus.cc:348] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } }
I20260811 06:16:51.202068 25146 raft_consensus.cc:374] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:51.202172 25146 raft_consensus.cc:729] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c971479a50a14aefa45ef7dfeac4c4a9, State: Initialized, Role: FOLLOWER
I20260811 06:16:51.202657 25146 consensus_queue.cc:260] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } }
I20260811 06:16:51.202867 25146 raft_consensus.cc:388] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:16:51.202986 25146 raft_consensus.cc:482] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:16:51.203143 25146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:51.205122 25146 raft_consensus.cc:504] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } }
I20260811 06:16:51.205596 25146 leader_election.cc:302] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c971479a50a14aefa45ef7dfeac4c4a9; no voters:
I20260811 06:16:51.206080 25146 leader_election.cc:288] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:16:51.206174 25156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:51.206779 25156 raft_consensus.cc:686] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [term 1 LEADER]: Becoming Leader. State: Replica: c971479a50a14aefa45ef7dfeac4c4a9, State: Running, Role: LEADER
I20260811 06:16:51.207245 25156 consensus_queue.cc:237] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } }
I20260811 06:16:51.207896 25146 sys_catalog.cc:564] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:16:51.209641 25158 sys_catalog.cc:455] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } } }
I20260811 06:16:51.209980 25158 sys_catalog.cc:458] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:51.215205 25159 sys_catalog.cc:455] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c971479a50a14aefa45ef7dfeac4c4a9. Latest consensus state: current_term: 1 leader_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c971479a50a14aefa45ef7dfeac4c4a9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46701 } } }
I20260811 06:16:51.215818 25159 sys_catalog.cc:458] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:51.216562 25166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:16:51.221830 25166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:16:51.229847 25166 catalog_manager.cc:1347] Generated new cluster ID: f0a48aedeef94301aa26dddc3385342f
I20260811 06:16:51.229945 25166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:16:51.238934 25166 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:16:51.240051 25166 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:16:51.250146 25166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Generated new TSK 0
I20260811 06:16:51.250772 25166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:16:51.304250 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46701
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:51.637681 25192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:51.637918 25192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:51.638049 25192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:51.644918 25192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:51.645016 25192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:51.645138 25192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:51.645237 25192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:16:51.654228 25192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:46701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:51.655283 25192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:51.656508 25192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:51.668246 25208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:51.668218 25205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:51.668583 25201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:51.668978 25192 server_base.cc:1034] running on GCE node
I20260811 06:16:51.669533 25192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:51.670308 25192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:51.671497 25192 hybrid_clock.cc:648] HybridClock initialized: now 1786429011671437 us; error 36 us; skew 500 ppm
I20260811 06:16:51.671761 25192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:51 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429011, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:51.674671 25192 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:16:51.676019 25192 webserver.cc:460] Webserver started at http://127.22.121.1:42025/ using document root <none> and password file <none>
I20260811 06:16:51.676630 25192 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:51.676730 25192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:51.677021 25192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:51.679723 25192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "7738cda8a42c496bb88cb24f94956ed4"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "d7acdff5853f602e589a575684370708"
server_key_iv: "3dfb74b7714392e6fbf13a1171bef1ca"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.680469 25192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "7738cda8a42c496bb88cb24f94956ed4"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "d7acdff5853f602e589a575684370708"
server_key_iv: "3dfb74b7714392e6fbf13a1171bef1ca"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.685091 25192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:16:51.688228 25224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:51.689366 25192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:16:51.689502 25192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "7738cda8a42c496bb88cb24f94956ed4"
format_stamp: "Formatted at 2026-08-11 06:16:51 on dist-test-slave-gjrz"
server_key: "d7acdff5853f602e589a575684370708"
server_key_iv: "3dfb74b7714392e6fbf13a1171bef1ca"
server_key_version: "encryptionkey@0"
I20260811 06:16:51.689703 25192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:51.714099 25192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:51.717928 25192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:51.718146 25192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:51.720675 25192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:51.720775 25192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:51.720964 25192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:51.721035 25192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:16:51 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429011, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:51.741346 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:51.720964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:39777 (local address 127.22.121.62:46701)
0811 06:16:51.721449 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:51.721457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:51.722561 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 06:16:51.724487 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:51.724493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:51.724574 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:51.724666 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:51.726534 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.727469 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.729841 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.730345 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.730474 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:51.735104 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:51.735110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:51.735115 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:51.735154 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:51.738002 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.738878 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.738883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.738887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.739033 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.739619 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.739626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.739630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.740009 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:51.740174 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:51.740791 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 06:16:51.740999 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260811 06:16:51.742202 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:51.721574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.1:39777)
0811 06:16:51.722405 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:51.722456 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:51.724232 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:51.724883 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:51.724891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:51.725513 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:51.726356 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:51.726375 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.727653 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.727658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:51.729633 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:51.729645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.730591 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.730601 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:51.730883 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:51.731652 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:51.731686 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:51.734898 (+ 3212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:51.738127 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:51.738136 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:51.738149 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:51.738718 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:51.739175 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:51.739179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:51.739181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:51.739473 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:51.740149 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:51.740155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:51.740437 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 06:16:51.740663 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 06:16:51.741112 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":319,"threads_started":1}
I20260811 06:16:51.760171 25192 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:36573
I20260811 06:16:51.760201 25425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:36573 every 8 connection(s)
I20260811 06:16:51.761332 25192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:16:51.762378 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:51.749751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:60701 (local address 127.22.121.62:46701)
0811 06:16:51.749905 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:51.749910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:51.750111 (+ 201us) server_negotiation.cc:407] Connection header received
0811 06:16:51.750360 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:51.750367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:51.750437 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:51.750487 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:51.751884 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.752564 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.754432 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.755068 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.755194 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:51.757388 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:51.757395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:51.757399 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:51.757435 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:51.759811 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.760618 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.760622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.760626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.760750 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:51.761062 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:51.761067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:51.761071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:51.761323 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:51.761456 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:51.761825 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 06:16:51.762047 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:16:51.762600 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:51.749339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.1:60701)
0811 06:16:51.750010 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:51.750037 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:51.750196 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:51.750664 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:51.750669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:51.751194 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:51.751666 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:51.751677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.752738 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.752743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:51.754285 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:51.754302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:51.755431 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:51.755441 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:51.755641 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:51.756173 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:51.756196 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:51.757169 (+ 973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:51.759925 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:51.759930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:51.759934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:51.760519 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:51.760845 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:51.760850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:51.760854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:51.760969 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:51.761431 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:51.761435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:51.761607 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 06:16:51.761721 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 06:16:51.762104 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":459,"threads_started":1}
I20260811 06:16:51.763762 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25192
I20260811 06:16:51.763957 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:16:51.764559 23012 external_mini_cluster.cc:1428] Setting key fd86f5dfaf154a0472b07d7cae1d2d22
I20260811 06:16:51.764094 25429 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46701
I20260811 06:16:51.765465 25429 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:51.766268 25429 heartbeater.cc:511] Master 127.22.121.62:46701 requested a full tablet report, sending...
I20260811 06:16:51.768370 25062 ts_manager.cc:194] Registered new tserver with Master: 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573)
I20260811 06:16:51.770138 25062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:39777
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:16:51.801918 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46701
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:52.106258 25435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:52.106519 25435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:52.106660 25435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:52.113673 25435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:52.113765 25435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:52.113852 25435 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:52.113926 25435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:16:52.123011 25435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:46701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:52.124125 25435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:52.125337 25435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:52.137476 25451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:52.137710 25443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:52.137634 25444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:52.138610 25435 server_base.cc:1034] running on GCE node
I20260811 06:16:52.139205 25435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:52.140149 25435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:52.141361 25435 hybrid_clock.cc:648] HybridClock initialized: now 1786429012141295 us; error 42 us; skew 500 ppm
I20260811 06:16:52.141630 25435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:52.148152 25435 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:16:52.149413 25435 webserver.cc:460] Webserver started at http://127.22.121.2:43571/ using document root <none> and password file <none>
I20260811 06:16:52.150044 25435 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:52.150144 25435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:52.150437 25435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:52.153215 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "1cf07357005c41909a4cd6a44e826f3c"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "7cb73afdf339d80afb0a9298b707c509"
server_key_iv: "9e45a8a074594088f9c9cda704261af6"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.153978 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "1cf07357005c41909a4cd6a44e826f3c"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "7cb73afdf339d80afb0a9298b707c509"
server_key_iv: "9e45a8a074594088f9c9cda704261af6"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.158766 25435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:16:52.162216 25460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:52.163465 25435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:16:52.163619 25435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "1cf07357005c41909a4cd6a44e826f3c"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "7cb73afdf339d80afb0a9298b707c509"
server_key_iv: "9e45a8a074594088f9c9cda704261af6"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.163815 25435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:52.183151 25435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:52.186633 25435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:52.186862 25435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:52.189397 25435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:52.189483 25435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:52.189623 25435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:52.189692 25435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:52.212311 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.189970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:51087 (local address 127.22.121.62:46701)
0811 06:16:52.190180 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.190186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.191524 (+ 1338us) server_negotiation.cc:407] Connection header received
0811 06:16:52.194184 (+ 2660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.194190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.194273 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.194372 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:52.196481 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.197593 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.199812 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.200369 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.200486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.205178 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:52.205183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:52.205186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:52.205219 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:52.208615 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.209686 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.209692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.209696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.209850 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.210260 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.210265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.210269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.210637 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:52.210782 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.211666 (+ 884us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.211921 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 06:16:52.212888 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.190466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.2:51087)
0811 06:16:52.191346 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.191406 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.193833 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.194727 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.194738 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.195484 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:52.196289 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.196317 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.197746 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.197752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.199679 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.199692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.200657 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.200670 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.200973 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.201725 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:52.201767 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:52.204979 (+ 3212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:52.208790 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.208799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.208813 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.209505 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.209961 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.209967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.209971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.210154 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.210774 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:52.210779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:52.211203 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.211538 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.211958 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":336,"threads_started":1}
I20260811 06:16:52.231683 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.217957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.2:52315)
0811 06:16:52.218495 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.218510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.218633 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.218903 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.218907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.219386 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:52.219816 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.219827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.220759 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.220768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.222323 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.222334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.223124 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.223132 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.223351 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.223984 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:52.224009 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:52.225350 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:52.228612 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.228621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.228625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.229142 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.229596 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.229601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.229605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.229771 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.230494 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:52.230499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:52.230661 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.230855 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.231160 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":306,"threads_started":1}
I20260811 06:16:52.232538 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.217966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:52315 (local address 127.22.121.62:46701)
0811 06:16:52.218111 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.218115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.218587 (+ 472us) server_negotiation.cc:407] Connection header received
0811 06:16:52.218736 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.218741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.218796 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.218868 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:52.219949 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.220634 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.222448 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.222905 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.223026 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.225562 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:52.225568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:52.225571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:52.225604 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:52.228478 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.229311 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.229317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.229320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.229477 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.229927 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.229933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.229936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.230273 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:52.230414 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.231985 (+ 1571us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.232209 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:16:52.233240 25435 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:32905
I20260811 06:16:52.233259 25680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:32905 every 8 connection(s)
I20260811 06:16:52.234694 25435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:16:52.234962 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25435
I20260811 06:16:52.235149 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:16:52.235734 23012 external_mini_cluster.cc:1428] Setting key 569d10d7d913f220d120b8b29d2def23
I20260811 06:16:52.236905 25681 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46701
I20260811 06:16:52.237123 25681 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:52.237867 25681 heartbeater.cc:511] Master 127.22.121.62:46701 requested a full tablet report, sending...
I20260811 06:16:52.239411 25062 ts_manager.cc:194] Registered new tserver with Master: 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905)
I20260811 06:16:52.240264 25062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:51087
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:16:52.275818 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46701
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:52.577492 25687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:52.577723 25687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:52.577818 25687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:52.584583 25687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:52.584672 25687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:52.584756 25687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:52.584833 25687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:16:52.593920 25687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:46701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:52.594945 25687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:52.596164 25687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:52.608074 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:52.608119 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:52.608567 25698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:52.608943 25687 server_base.cc:1034] running on GCE node
I20260811 06:16:52.609530 25687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:52.610351 25687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:52.611554 25687 hybrid_clock.cc:648] HybridClock initialized: now 1786429012611489 us; error 42 us; skew 500 ppm
I20260811 06:16:52.611828 25687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:52.614678 25687 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:16:52.616006 25687 webserver.cc:460] Webserver started at http://127.22.121.3:45353/ using document root <none> and password file <none>
I20260811 06:16:52.616621 25687 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:52.616719 25687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:52.617026 25687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:52.619743 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "50ba83070b144bae81a498f85b88b085"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "11cd0b0131a1496da003c77b5887666f"
server_key_iv: "375b230c6a1429833dc3b31c48fcb492"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.620568 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "50ba83070b144bae81a498f85b88b085"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "11cd0b0131a1496da003c77b5887666f"
server_key_iv: "375b230c6a1429833dc3b31c48fcb492"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.625382 25687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:16:52.628518 25713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:52.629678 25687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:16:52.629825 25687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "50ba83070b144bae81a498f85b88b085"
format_stamp: "Formatted at 2026-08-11 06:16:52 on dist-test-slave-gjrz"
server_key: "11cd0b0131a1496da003c77b5887666f"
server_key_iv: "375b230c6a1429833dc3b31c48fcb492"
server_key_version: "encryptionkey@0"
I20260811 06:16:52.630008 25687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:52.645681 25687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:52.649139 25687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:52.649367 25687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:52.651881 25687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:52.651983 25687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:52.652190 25687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:52.652292 25687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:52.673966 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.652874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.3:58615)
0811 06:16:52.653636 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.653675 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.655469 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.656145 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.656156 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.656824 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:52.657607 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.657640 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.659097 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.659103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.661167 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.661186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.662218 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.662224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.662394 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.663003 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:52.663036 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:52.666061 (+ 3025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:52.669406 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.669418 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.669432 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.669994 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.670431 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.670436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.670441 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.670718 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.671457 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:52.671463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:52.671883 (+ 420us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.672139 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.672618 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":326,"threads_started":1}
I20260811 06:16:52.674868 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.652229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58615 (local address 127.22.121.62:46701)
0811 06:16:52.652401 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.652407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.653782 (+ 1375us) server_negotiation.cc:407] Connection header received
0811 06:16:52.655742 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.655748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.655829 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.655913 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:52.657842 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.658905 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.661325 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.661994 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.662152 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.666335 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:52.666342 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:52.666346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:52.666382 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:52.669281 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.670165 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.670171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.670174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.670324 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.670875 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.670878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.670880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.671216 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:52.671393 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.674466 (+ 3073us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.674632 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:16:52.692528 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.680336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:59401 (local address 127.22.121.62:46701)
0811 06:16:52.680496 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.680502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.681108 (+ 606us) server_negotiation.cc:407] Connection header received
0811 06:16:52.681270 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.681274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.681325 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.681391 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:52.682527 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.683160 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.685259 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.685661 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.685759 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.687843 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:52.687848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:52.687851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:52.687878 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:52.690080 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.690659 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.690665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.690668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.690784 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.691100 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.691105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.691109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.691401 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:52.691525 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.692034 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.692210 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:16:52.693665 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.680367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.22.121.3:59401)
0811 06:16:52.681000 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.681021 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.681166 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.681432 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.681437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.682026 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:52.682408 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.682420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.683293 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.683298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.685127 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.685142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.685839 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.685846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.686046 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.686667 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:52.686684 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:52.687696 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:52.690196 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.690201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.690205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.690556 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.690885 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:52.690889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:52.690892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:52.691004 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:52.691515 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:52.691520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:52.691732 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.692899 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.693193 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":320,"threads_started":1}
I20260811 06:16:52.697546 25687 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41337
I20260811 06:16:52.697573 25936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41337 every 8 connection(s)
I20260811 06:16:52.698652 25687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:16:52.699293 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25687
I20260811 06:16:52.699491 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:16:52.700011 23012 external_mini_cluster.cc:1428] Setting key 3be7212b1b8b63478a29ed5172ad4c45
I20260811 06:16:52.700423 25941 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46701
I20260811 06:16:52.700801 25941 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:52.701413 25941 heartbeater.cc:511] Master 127.22.121.62:46701 requested a full tablet report, sending...
I20260811 06:16:52.702802 25061 ts_manager.cc:194] Registered new tserver with Master: 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337)
I20260811 06:16:52.703663 25061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:58615
I20260811 06:16:52.714566 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:16:52 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:52.759992 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.746048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.22.121.62:46701)
0811 06:16:52.746257 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.746263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.746524 (+ 261us) server_negotiation.cc:407] Connection header received
0811 06:16:52.746678 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.746681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.746756 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.746831 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:52.748115 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.748815 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.750343 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.750755 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.750862 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.754359 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:52.754365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:52.754369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:52.754409 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:52.757363 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.757951 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.757957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.757961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.758093 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:52.758443 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:52.758448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:52.758452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:52.758717 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:52.758954 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.759300 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.759633 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:16:52.772753 25429 heartbeater.cc:503] Master 127.22.121.62:46701 was elected leader, sending a full tablet report...
I20260811 06:16:52.770411 25061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:16:52.778689 25061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:52.804997 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.792266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36573 (local address 127.0.0.1:38152)
0811 06:16:52.793423 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.793472 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.793633 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.796772 (+ 3139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.796781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.796816 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:52.797255 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.797268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.798906 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.798913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.801329 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.801345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.804106 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.804113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.804312 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.804362 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.804461 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.804640 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":11010,"threads_started":1}
I20260811 06:16:52.806433 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.792438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.22.121.1:36573)
0811 06:16:52.796294 (+ 3856us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.796300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.796322 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:16:52.796394 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.796399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.796565 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.796711 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:52.797442 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.798798 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.801478 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.803868 (+ 2390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.805726 (+ 1858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.805919 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.805986 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.806157 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3725,"thread_start_us":3579,"threads_started":1}
I20260811 06:16:52.807914 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.793087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.22.121.3:41337)
0811 06:16:52.795854 (+ 2767us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.795864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.799915 (+ 4051us) server_negotiation.cc:407] Connection header received
0811 06:16:52.800020 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.800028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.800234 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.800422 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:52.800980 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.802768 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.805244 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.806639 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.806760 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.806957 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.807407 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.807607 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2548,"thread_start_us":2710,"threads_started":1}
I20260811 06:16:52.807881 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.792789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41337 (local address 127.0.0.1:33120)
0811 06:16:52.799668 (+ 6879us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.799688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.799797 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.800403 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.800407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.800432 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:52.800794 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.800804 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.802921 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.802926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.805065 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.805081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.807076 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.807081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.807202 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.807223 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.807334 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.807463 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6766,"thread_start_us":244,"threads_started":1}
I20260811 06:16:52.811700 25804 tablet_service.cc:1467] Processing CreateTablet for tablet ae609d0042334b62b8890697efa0bc29 (DEFAULT_TABLE table=test-table [id=a459ae8d89e04896aaf5f8d2655de19f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:52.813167 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae609d0042334b62b8890697efa0bc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:52.819844 25306 tablet_service.cc:1467] Processing CreateTablet for tablet ae609d0042334b62b8890697efa0bc29 (DEFAULT_TABLE table=test-table [id=a459ae8d89e04896aaf5f8d2655de19f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:52.821189 25306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae609d0042334b62b8890697efa0bc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:52.825675 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.803660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32905 (local address 127.0.0.1:38846)
0811 06:16:52.805464 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:52.805480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:52.805567 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:52.818553 (+ 12986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:52.818558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:52.818585 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:52.818746 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.818758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.821522 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.821527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.823291 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:52.823308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.824944 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.824950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:52.825144 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.825177 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:16:52.825273 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:16:52.825402 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"thread_start_us":12887,"threads_started":1}
I20260811 06:16:52.825791 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:52.803906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.22.121.2:32905)
0811 06:16:52.817956 (+ 14050us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:52.817966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:52.817996 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:16:52.818090 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:52.818098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:52.818334 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:52.818547 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:52.818928 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.821355 (+ 2427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.823500 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:52.824757 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:52.824895 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:52.825084 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:52.825338 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 06:16:52.825502 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13879,"thread_start_us":14782,"threads_started":1}
I20260811 06:16:52.828224 25978 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Bootstrap starting.
I20260811 06:16:52.829211 25540 tablet_service.cc:1467] Processing CreateTablet for tablet ae609d0042334b62b8890697efa0bc29 (DEFAULT_TABLE table=test-table [id=a459ae8d89e04896aaf5f8d2655de19f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:52.830329 25540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae609d0042334b62b8890697efa0bc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:52.832751 25978 tablet_bootstrap.cc:654] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:52.834115 25978 log.cc:824] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:52.834929 25979 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Bootstrap starting.
I20260811 06:16:52.838377 25979 tablet_bootstrap.cc:654] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:52.840375 25979 log.cc:824] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:52.841711 25978 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: No bootstrap required, opened a new log
I20260811 06:16:52.841970 25978 ts_tablet_manager.cc:1397] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.010s
I20260811 06:16:52.843436 25979 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: No bootstrap required, opened a new log
I20260811 06:16:52.843750 25979 ts_tablet_manager.cc:1397] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260811 06:16:52.843892 25983 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Bootstrap starting.
I20260811 06:16:52.847754 25983 tablet_bootstrap.cc:654] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:52.849282 25983 log.cc:824] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:52.851668 25979 raft_consensus.cc:348] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.852097 25979 raft_consensus.cc:374] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:52.852159 25983 tablet_bootstrap.cc:492] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: No bootstrap required, opened a new log
I20260811 06:16:52.852252 25979 raft_consensus.cc:729] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7738cda8a42c496bb88cb24f94956ed4, State: Initialized, Role: FOLLOWER
I20260811 06:16:52.852504 25983 ts_tablet_manager.cc:1397] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:16:52.852404 25978 raft_consensus.cc:348] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.852843 25978 raft_consensus.cc:374] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:52.852981 25978 raft_consensus.cc:729] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ba83070b144bae81a498f85b88b085, State: Initialized, Role: FOLLOWER
I20260811 06:16:52.853041 25979 consensus_queue.cc:260] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.853597 25978 consensus_queue.cc:260] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.854683 25941 heartbeater.cc:503] Master 127.22.121.62:46701 was elected leader, sending a full tablet report...
I20260811 06:16:52.855376 25978 ts_tablet_manager.cc:1428] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Time spent starting tablet: real 0.013s user 0.008s sys 0.007s
I20260811 06:16:52.860317 25979 ts_tablet_manager.cc:1428] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Time spent starting tablet: real 0.016s user 0.010s sys 0.003s
I20260811 06:16:52.862254 25983 raft_consensus.cc:348] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.862653 25983 raft_consensus.cc:374] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:52.862758 25983 raft_consensus.cc:729] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf07357005c41909a4cd6a44e826f3c, State: Initialized, Role: FOLLOWER
I20260811 06:16:52.863373 25983 consensus_queue.cc:260] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:52.864192 25681 heartbeater.cc:503] Master 127.22.121.62:46701 was elected leader, sending a full tablet report...
I20260811 06:16:52.864602 25983 ts_tablet_manager.cc:1428] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260811 06:16:52.950132 25942 tablet.cc:2369] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:16:52.990496 25684 tablet.cc:2369] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:16:53.016173 25430 tablet.cc:2369] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:16:53.079200 25988 raft_consensus.cc:482] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:53.079550 25988 raft_consensus.cc:504] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.081410 25988 leader_election.cc:288] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573), 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337)
I20260811 06:16:53.088828 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.081929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55777 (local address 127.22.121.1:36573)
0811 06:16:53.082169 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.082176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.082815 (+ 639us) server_negotiation.cc:407] Connection header received
0811 06:16:53.082992 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.082998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.083072 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.083153 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.083946 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.084680 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.086907 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.087980 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.088136 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.088212 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.088478 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.088601 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:16:53.088986 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.081702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36573 (local address 127.22.121.2:55777)
0811 06:16:53.082714 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.082732 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.082867 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.083193 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.083197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.083225 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.083806 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.083818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.084811 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.084818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.086754 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.086769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.088191 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.088198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.088335 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.088359 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.088440 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.088561 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":19,"thread_start_us":689,"threads_started":1}
I20260811 06:16:53.089401 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.082307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41337 (local address 127.22.121.2:34119)
0811 06:16:53.083737 (+ 1430us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.083752 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.083911 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.084217 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.084221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.084246 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.084866 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.084878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.086042 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.086047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.087476 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.087488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.088588 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.088594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.088729 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.088751 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.088832 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.088962 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":3,"thread_start_us":847,"threads_started":1}
I20260811 06:16:53.090138 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.082129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:34119 (local address 127.22.121.3:41337)
0811 06:16:53.082341 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.082346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.083845 (+ 1499us) server_negotiation.cc:407] Connection header received
0811 06:16:53.084044 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.084048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.084111 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.084184 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.085051 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.085899 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.087626 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.088373 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.088527 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.088607 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.088872 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.088990 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:16:53.090144 25344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae609d0042334b62b8890697efa0bc29" candidate_uuid: "1cf07357005c41909a4cd6a44e826f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7738cda8a42c496bb88cb24f94956ed4" is_pre_election: true
I20260811 06:16:53.090705 25344 raft_consensus.cc:2454] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf07357005c41909a4cd6a44e826f3c in term 0.
I20260811 06:16:53.090983 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae609d0042334b62b8890697efa0bc29" candidate_uuid: "1cf07357005c41909a4cd6a44e826f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ba83070b144bae81a498f85b88b085" is_pre_election: true
I20260811 06:16:53.091348 25462 leader_election.cc:302] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf07357005c41909a4cd6a44e826f3c, 7738cda8a42c496bb88cb24f94956ed4; no voters:
I20260811 06:16:53.091508 25847 raft_consensus.cc:2454] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf07357005c41909a4cd6a44e826f3c in term 0.
I20260811 06:16:53.091686 25988 raft_consensus.cc:2781] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:53.091832 25988 raft_consensus.cc:482] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:53.091946 25988 raft_consensus.cc:3037] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.094148 25988 raft_consensus.cc:504] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.094667 25988 leader_election.cc:288] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [CANDIDATE]: Term 1 election: Requested vote from peers 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573), 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337)
I20260811 06:16:53.095017 25344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae609d0042334b62b8890697efa0bc29" candidate_uuid: "1cf07357005c41909a4cd6a44e826f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7738cda8a42c496bb88cb24f94956ed4"
I20260811 06:16:53.095146 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae609d0042334b62b8890697efa0bc29" candidate_uuid: "1cf07357005c41909a4cd6a44e826f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ba83070b144bae81a498f85b88b085"
I20260811 06:16:53.095237 25344 raft_consensus.cc:3037] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.095355 25847 raft_consensus.cc:3037] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.098747 25344 raft_consensus.cc:2454] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf07357005c41909a4cd6a44e826f3c in term 1.
I20260811 06:16:53.098747 25847 raft_consensus.cc:2454] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf07357005c41909a4cd6a44e826f3c in term 1.
I20260811 06:16:53.099133 25462 leader_election.cc:302] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf07357005c41909a4cd6a44e826f3c, 7738cda8a42c496bb88cb24f94956ed4; no voters:
I20260811 06:16:53.099419 25988 raft_consensus.cc:2781] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:53.099926 25988 raft_consensus.cc:686] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 1 LEADER]: Becoming Leader. State: Replica: 1cf07357005c41909a4cd6a44e826f3c, State: Running, Role: LEADER
I20260811 06:16:53.100490 25988 consensus_queue.cc:237] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.105013 25062 catalog_manager.cc:5495] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2). New cstate: current_term: 1 leader_uuid: "1cf07357005c41909a4cd6a44e826f3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } health_report { overall_health: UNKNOWN } } }
I20260811 06:16:53.174652 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.168499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.22.121.2:32905)
0811 06:16:53.168715 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.168722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.168990 (+ 268us) server_negotiation.cc:407] Connection header received
0811 06:16:53.169289 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.169293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.169380 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.169507 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:53.170163 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.170901 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.172512 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.172909 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.173021 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.173479 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.174027 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.174162 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.174290 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.174373 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:16:53.184870 25344 raft_consensus.cc:1261] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Refusing update from remote peer 1cf07357005c41909a4cd6a44e826f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.185870 25994 consensus_queue.cc:1035] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:53.186820 25847 raft_consensus.cc:1261] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Refusing update from remote peer 1cf07357005c41909a4cd6a44e826f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.187669 25994 consensus_queue.cc:1035] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:53.197467 25999 mvcc.cc:205] Tried to move back new op lower bound from 7317213237990354944 to 7317213237663023104. Current Snapshot: MvccSnapshot[applied={T|T < 7317213237990354944}]
I20260811 06:16:53.221396 25062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:53.221736 25062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:53.225982 25062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Sending DeleteTablet for 3 replicas of tablet ae609d0042334b62b8890697efa0bc29
I20260811 06:16:53.227327 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet ae609d0042334b62b8890697efa0bc29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33120
I20260811 06:16:53.227350 25306 tablet_service.cc:1514] Processing DeleteTablet for tablet ae609d0042334b62b8890697efa0bc29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260811 06:16:53.227819 25540 tablet_service.cc:1514] Processing DeleteTablet for tablet ae609d0042334b62b8890697efa0bc29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38846
I20260811 06:16:53.229120 25062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:16:53.229756 25062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:53.239130 25799 tablet_service.cc:1467] Processing CreateTablet for tablet 3af5d1fa2e034f879d661a5813672c67 (DEFAULT_TABLE table=test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:53.239799 25799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af5d1fa2e034f879d661a5813672c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.240363 25305 tablet_service.cc:1467] Processing CreateTablet for tablet 3af5d1fa2e034f879d661a5813672c67 (DEFAULT_TABLE table=test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:53.241262 25305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af5d1fa2e034f879d661a5813672c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.249198 25978 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Bootstrap starting.
I20260811 06:16:53.249929 25979 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Bootstrap starting.
I20260811 06:16:53.243512 25539 tablet_service.cc:1467] Processing CreateTablet for tablet 3af5d1fa2e034f879d661a5813672c67 (DEFAULT_TABLE table=test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:53.251495 25978 tablet_bootstrap.cc:654] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.251751 25539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af5d1fa2e034f879d661a5813672c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.252170 25979 tablet_bootstrap.cc:654] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.262238 26022 tablet_replica.cc:330] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: stopping tablet replica
I20260811 06:16:53.263109 26019 tablet_replica.cc:330] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: stopping tablet replica
I20260811 06:16:53.264050 26022 raft_consensus.cc:2229] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:16:53.264158 25978 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: No bootstrap required, opened a new log
I20260811 06:16:53.264314 25978 ts_tablet_manager.cc:1397] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260811 06:16:53.265015 26019 raft_consensus.cc:2229] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:53.265959 26022 raft_consensus.cc:2258] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:53.266255 25978 raft_consensus.cc:348] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.266572 25978 raft_consensus.cc:374] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.266724 25978 raft_consensus.cc:729] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ba83070b144bae81a498f85b88b085, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.267102 25978 consensus_queue.cc:260] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.268566 26022 ts_tablet_manager.cc:1905] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:53.269071 25983 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Bootstrap starting.
I20260811 06:16:53.269090 25978 ts_tablet_manager.cc:1428] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 06:16:53.271013 25983 tablet_bootstrap.cc:654] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.271874 26017 tablet_replica.cc:330] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: stopping tablet replica
I20260811 06:16:53.267442 25979 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: No bootstrap required, opened a new log
I20260811 06:16:53.273058 25979 ts_tablet_manager.cc:1397] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.000s
I20260811 06:16:53.273876 25979 raft_consensus.cc:348] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.274092 25979 raft_consensus.cc:374] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.274266 25979 raft_consensus.cc:729] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7738cda8a42c496bb88cb24f94956ed4, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.274740 25979 consensus_queue.cc:260] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.276500 25979 ts_tablet_manager.cc:1428] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:16:53.281451 26019 raft_consensus.cc:2258] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:53.282090 26017 raft_consensus.cc:2229] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:53.283972 26017 raft_consensus.cc:2258] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:53.285181 26022 ts_tablet_manager.cc:1918] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:16:53.285943 26019 ts_tablet_manager.cc:1905] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:53.287770 26017 ts_tablet_manager.cc:1905] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:53.294200 26022 log.cc:1196] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/ae609d0042334b62b8890697efa0bc29
I20260811 06:16:53.294811 26022 ts_tablet_manager.cc:1939] T ae609d0042334b62b8890697efa0bc29 P 1cf07357005c41909a4cd6a44e826f3c: Deleting consensus metadata
I20260811 06:16:53.294888 25983 tablet_bootstrap.cc:492] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: No bootstrap required, opened a new log
I20260811 06:16:53.295074 25983 ts_tablet_manager.cc:1397] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.016s
I20260811 06:16:53.296241 25983 raft_consensus.cc:348] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.296469 25983 raft_consensus.cc:374] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.296569 25983 raft_consensus.cc:729] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf07357005c41909a4cd6a44e826f3c, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.297457 26019 ts_tablet_manager.cc:1918] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:16:53.297693 26019 log.cc:1196] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/ae609d0042334b62b8890697efa0bc29
I20260811 06:16:53.298461 26019 ts_tablet_manager.cc:1939] T ae609d0042334b62b8890697efa0bc29 P 7738cda8a42c496bb88cb24f94956ed4: Deleting consensus metadata
I20260811 06:16:53.300485 25038 catalog_manager.cc:4841] TS 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573): tablet ae609d0042334b62b8890697efa0bc29 (table test-table [id=a459ae8d89e04896aaf5f8d2655de19f]) successfully deleted
I20260811 06:16:53.301462 25983 consensus_queue.cc:260] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.302536 26017 ts_tablet_manager.cc:1918] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:16:53.302619 25038 catalog_manager.cc:4841] TS 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905): tablet ae609d0042334b62b8890697efa0bc29 (table test-table [id=a459ae8d89e04896aaf5f8d2655de19f]) successfully deleted
I20260811 06:16:53.303277 26017 log.cc:1196] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/ae609d0042334b62b8890697efa0bc29
I20260811 06:16:53.303274 25983 ts_tablet_manager.cc:1428] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260811 06:16:53.303862 26017 ts_tablet_manager.cc:1939] T ae609d0042334b62b8890697efa0bc29 P 50ba83070b144bae81a498f85b88b085: Deleting consensus metadata
I20260811 06:16:53.305603 25041 catalog_manager.cc:4841] TS 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337): tablet ae609d0042334b62b8890697efa0bc29 (table test-table [id=a459ae8d89e04896aaf5f8d2655de19f]) successfully deleted
I20260811 06:16:53.529887 25986 raft_consensus.cc:482] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:53.530113 25986 raft_consensus.cc:504] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.532902 25986 leader_election.cc:288] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905), 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337)
I20260811 06:16:53.541903 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.532912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32905 (local address 127.22.121.1:59239)
0811 06:16:53.533983 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.534028 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.534314 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.534904 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.534908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.534932 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.535403 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.535416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.537101 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.537105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.539490 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.539502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.541067 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.541074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.541219 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.541244 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.541374 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.541512 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":763,"threads_started":1}
I20260811 06:16:53.541978 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.533259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:59239 (local address 127.22.121.2:32905)
0811 06:16:53.533451 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.533457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.534132 (+ 675us) server_negotiation.cc:407] Connection header received
0811 06:16:53.534532 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.534539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.534645 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.534706 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.535606 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.536947 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.539651 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.540747 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.540891 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.540975 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.541413 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.541549 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:16:53.543325 25581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af5d1fa2e034f879d661a5813672c67" candidate_uuid: "7738cda8a42c496bb88cb24f94956ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf07357005c41909a4cd6a44e826f3c" is_pre_election: true
I20260811 06:16:53.544085 25581 raft_consensus.cc:2454] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7738cda8a42c496bb88cb24f94956ed4 in term 0.
I20260811 06:16:53.544317 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.533351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41337 (local address 127.22.121.1:51689)
0811 06:16:53.535030 (+ 1679us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.535048 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.535211 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.537147 (+ 1936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.537152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.537183 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.537838 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.537856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.539332 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.539338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.541541 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.541558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.543471 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.543479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.543671 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.543728 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.543867 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.544019 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":1312,"threads_started":1}
I20260811 06:16:53.544548 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.533072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:51689 (local address 127.22.121.3:41337)
0811 06:16:53.533270 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.533277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.535173 (+ 1896us) server_negotiation.cc:407] Connection header received
0811 06:16:53.535365 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.535370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.535461 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.535560 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.538026 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.539158 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.541745 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.543219 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.543392 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.543465 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.543921 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.544116 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:16:53.545049 25229 leader_election.cc:302] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf07357005c41909a4cd6a44e826f3c, 7738cda8a42c496bb88cb24f94956ed4; no voters:
I20260811 06:16:53.545192 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af5d1fa2e034f879d661a5813672c67" candidate_uuid: "7738cda8a42c496bb88cb24f94956ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ba83070b144bae81a498f85b88b085" is_pre_election: true
I20260811 06:16:53.545423 25847 raft_consensus.cc:2454] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7738cda8a42c496bb88cb24f94956ed4 in term 0.
I20260811 06:16:53.545475 25986 raft_consensus.cc:2781] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:53.545631 25986 raft_consensus.cc:482] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:53.545756 25986 raft_consensus.cc:3037] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.549701 25986 raft_consensus.cc:504] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.550437 25986 leader_election.cc:288] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [CANDIDATE]: Term 1 election: Requested vote from peers 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905), 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337)
I20260811 06:16:53.550864 25581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af5d1fa2e034f879d661a5813672c67" candidate_uuid: "7738cda8a42c496bb88cb24f94956ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf07357005c41909a4cd6a44e826f3c"
I20260811 06:16:53.550906 25847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af5d1fa2e034f879d661a5813672c67" candidate_uuid: "7738cda8a42c496bb88cb24f94956ed4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ba83070b144bae81a498f85b88b085"
I20260811 06:16:53.551092 25847 raft_consensus.cc:3037] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.551092 25581 raft_consensus.cc:3037] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.553323 25847 raft_consensus.cc:2454] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7738cda8a42c496bb88cb24f94956ed4 in term 1.
I20260811 06:16:53.553905 25233 leader_election.cc:302] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ba83070b144bae81a498f85b88b085, 7738cda8a42c496bb88cb24f94956ed4; no voters:
I20260811 06:16:53.554435 25986 raft_consensus.cc:2781] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:53.555233 25581 raft_consensus.cc:2454] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7738cda8a42c496bb88cb24f94956ed4 in term 1.
I20260811 06:16:53.564643 25986 raft_consensus.cc:686] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 7738cda8a42c496bb88cb24f94956ed4, State: Running, Role: LEADER
I20260811 06:16:53.565016 25986 consensus_queue.cc:237] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.568831 25062 catalog_manager.cc:5495] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "7738cda8a42c496bb88cb24f94956ed4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:16:53 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429011, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:53.670269 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.653824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.22.121.62:46701)
0811 06:16:53.654526 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.654534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.654971 (+ 437us) server_negotiation.cc:407] Connection header received
0811 06:16:53.655150 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.655154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.655209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.655334 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:53.656923 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.657736 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.659616 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.660026 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.660115 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.664129 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:53.664134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:53.664137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:53.664165 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:53.667453 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:53.668298 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:53.668321 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:53.668325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:53.668475 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:53.668903 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:53.668906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:53.668908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:53.669167 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:53.669270 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.669785 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.669947 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":323,"threads_started":1}
I20260811 06:16:53.670567 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.653838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46701 (local address 127.0.0.1:37438)
0811 06:16:53.654857 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.654884 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.655041 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.655367 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.655376 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.656406 (+ 1030us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:53.656760 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.656775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.657885 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.657891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.659478 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.659494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.660202 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.660214 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.660409 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.661076 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:53.661104 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:53.663940 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:53.667584 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:53.667597 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:53.667606 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:53.668194 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:53.668590 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:53.668595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:53.668598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:53.668800 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:53.669287 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:53.669296 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:53.669511 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.669722 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.670122 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":104,"thread_start_us":746,"threads_started":1}
I20260811 06:16:53.675976 25062 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:37438:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:16:53.677434 25062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:53.684016 25306 tablet_service.cc:1467] Processing CreateTablet for tablet 2eeeec11a41946799f1154efad0dc995 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cdb17244fb646d1a64811335040e583]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:53.684360 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 2eeeec11a41946799f1154efad0dc995 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cdb17244fb646d1a64811335040e583]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:53.684674 25306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eeeec11a41946799f1154efad0dc995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.684824 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eeeec11a41946799f1154efad0dc995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.690948 25978 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: Bootstrap starting.
I20260811 06:16:53.690774 25539 tablet_service.cc:1467] Processing CreateTablet for tablet 2eeeec11a41946799f1154efad0dc995 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cdb17244fb646d1a64811335040e583]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:53.691355 25539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eeeec11a41946799f1154efad0dc995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:53.692807 25978 tablet_bootstrap.cc:654] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.692975 25979 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: Bootstrap starting.
I20260811 06:16:53.695009 25979 tablet_bootstrap.cc:654] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.702284 25983 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: Bootstrap starting.
I20260811 06:16:53.704448 25979 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: No bootstrap required, opened a new log
I20260811 06:16:53.704690 25979 ts_tablet_manager.cc:1397] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260811 06:16:53.705284 25983 tablet_bootstrap.cc:654] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:53.705686 25979 raft_consensus.cc:348] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.706003 25979 raft_consensus.cc:374] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.706133 25979 raft_consensus.cc:729] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7738cda8a42c496bb88cb24f94956ed4, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.706480 25979 consensus_queue.cc:260] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.707139 25986 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.708422 25986 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.710649 25979 ts_tablet_manager.cc:1428] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 06:16:53.711233 25983 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: No bootstrap required, opened a new log
I20260811 06:16:53.711493 25983 ts_tablet_manager.cc:1397] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:16:53.712471 25983 raft_consensus.cc:348] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.712786 25983 raft_consensus.cc:374] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.712939 25983 raft_consensus.cc:729] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf07357005c41909a4cd6a44e826f3c, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.713328 25983 consensus_queue.cc:260] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.713702 25988 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.714027 25988 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.714581 25983 ts_tablet_manager.cc:1428] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:16:53.716015 25978 tablet_bootstrap.cc:492] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: No bootstrap required, opened a new log
I20260811 06:16:53.716382 25978 ts_tablet_manager.cc:1397] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.007s
I20260811 06:16:53.718386 25978 raft_consensus.cc:348] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.718694 25978 raft_consensus.cc:374] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:53.718839 25978 raft_consensus.cc:729] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ba83070b144bae81a498f85b88b085, State: Initialized, Role: FOLLOWER
I20260811 06:16:53.719164 25978 consensus_queue.cc:260] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.719744 25987 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.720413 25987 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.721058 25978 ts_tablet_manager.cc:1428] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:16:53.821611 25987 raft_consensus.cc:482] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:53.821812 25987 raft_consensus.cc:504] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.823361 25987 leader_election.cc:288] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573), 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905)
I20260811 06:16:53.829957 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.824302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:39169 (local address 127.22.121.2:32905)
0811 06:16:53.824440 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.824445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.824725 (+ 280us) server_negotiation.cc:407] Connection header received
0811 06:16:53.824875 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.824881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.824954 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.825038 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.825597 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.826340 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.828184 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.828959 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.829081 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.829132 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.829478 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.829625 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 06:16:53.829952 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.824146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32905 (local address 127.22.121.3:39169)
0811 06:16:53.824616 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.824633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.824755 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.825067 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.825084 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.825108 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.825444 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.825457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.826467 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.826470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.828014 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.828030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.829119 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.829125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.829303 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.829344 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.829439 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.829540 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":5,"thread_start_us":295,"threads_started":1}
I20260811 06:16:53.830197 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.823621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:41615 (local address 127.22.121.1:36573)
0811 06:16:53.824113 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.824119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.824139 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:16:53.824289 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.824294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.824372 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.824474 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:53.825077 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.825976 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.827796 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.829116 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.829338 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.829407 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.829737 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.829894 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":229,"threads_started":1}
I20260811 06:16:53.830308 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.823519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36573 (local address 127.22.121.3:41615)
0811 06:16:53.824002 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.824018 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.824150 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.824517 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.824522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.824561 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:53.824913 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.824927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.826115 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.826121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.827654 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.827666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.829350 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.829355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.829532 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.829565 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.829661 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.829793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":20,"thread_start_us":261,"threads_started":1}
I20260811 06:16:53.830626 25581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eeeec11a41946799f1154efad0dc995" candidate_uuid: "50ba83070b144bae81a498f85b88b085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf07357005c41909a4cd6a44e826f3c" is_pre_election: true
I20260811 06:16:53.830863 25581 raft_consensus.cc:2454] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ba83070b144bae81a498f85b88b085 in term 0.
I20260811 06:16:53.830953 25344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eeeec11a41946799f1154efad0dc995" candidate_uuid: "50ba83070b144bae81a498f85b88b085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7738cda8a42c496bb88cb24f94956ed4" is_pre_election: true
I20260811 06:16:53.831202 25344 raft_consensus.cc:2454] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ba83070b144bae81a498f85b88b085 in term 0.
I20260811 06:16:53.831386 25718 leader_election.cc:302] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf07357005c41909a4cd6a44e826f3c, 50ba83070b144bae81a498f85b88b085; no voters:
I20260811 06:16:53.831769 25987 raft_consensus.cc:2781] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:53.831892 25987 raft_consensus.cc:482] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:53.831987 25987 raft_consensus.cc:3037] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.834185 25987 raft_consensus.cc:504] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.834641 25987 leader_election.cc:288] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [CANDIDATE]: Term 1 election: Requested vote from peers 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573), 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905)
I20260811 06:16:53.835103 25344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eeeec11a41946799f1154efad0dc995" candidate_uuid: "50ba83070b144bae81a498f85b88b085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7738cda8a42c496bb88cb24f94956ed4"
I20260811 06:16:53.835125 25581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eeeec11a41946799f1154efad0dc995" candidate_uuid: "50ba83070b144bae81a498f85b88b085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf07357005c41909a4cd6a44e826f3c"
I20260811 06:16:53.835286 25344 raft_consensus.cc:3037] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.835315 25581 raft_consensus.cc:3037] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:53.837147 25344 raft_consensus.cc:2454] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ba83070b144bae81a498f85b88b085 in term 1.
I20260811 06:16:53.837194 25581 raft_consensus.cc:2454] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ba83070b144bae81a498f85b88b085 in term 1.
I20260811 06:16:53.837498 25718 leader_election.cc:302] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf07357005c41909a4cd6a44e826f3c, 50ba83070b144bae81a498f85b88b085; no voters:
I20260811 06:16:53.837755 25987 raft_consensus.cc:2781] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:53.838296 25987 raft_consensus.cc:686] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [term 1 LEADER]: Becoming Leader. State: Replica: 50ba83070b144bae81a498f85b88b085, State: Running, Role: LEADER
I20260811 06:16:53.838702 25987 consensus_queue.cc:237] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } }
I20260811 06:16:53.840694 26061 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: TxnStatusTablet state changed. Reason: New leader 50ba83070b144bae81a498f85b88b085. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.841118 26061 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:53.841920 26064 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:16:53.842345 26064 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:16:53.842355 25060 catalog_manager.cc:5495] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50ba83070b144bae81a498f85b88b085 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } health_report { overall_health: HEALTHY } } }
I20260811 06:16:53.889832 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.883791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.22.121.3:41337)
0811 06:16:53.883962 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.883968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.884054 (+ 86us) server_negotiation.cc:407] Connection header received
0811 06:16:53.884317 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.884320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.884400 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.884508 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:53.884890 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.885617 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.887685 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.888131 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.888216 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.888809 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.889264 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.889359 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.889531 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.889614 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:16:53.889976 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.883730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41337 (local address 127.0.0.1:33134)
0811 06:16:53.883968 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.883988 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.884141 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.884556 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.884562 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.884575 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:53.884762 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.884774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.885753 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.885759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.887552 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.887568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.888285 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.888291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.888558 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.888638 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.889376 (+ 738us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.889424 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.889522 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.889680 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:16:53.894497 25344 raft_consensus.cc:1261] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Refusing update from remote peer 50ba83070b144bae81a498f85b88b085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.894797 25581 raft_consensus.cc:1261] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Refusing update from remote peer 50ba83070b144bae81a498f85b88b085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.895191 26061 consensus_queue.cc:1035] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:53.895756 25987 consensus_queue.cc:1035] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:53.900223 25986 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: TxnStatusTablet state changed. Reason: New leader 50ba83070b144bae81a498f85b88b085. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.900540 25986 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.902194 25988 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: TxnStatusTablet state changed. Reason: New leader 50ba83070b144bae81a498f85b88b085. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.902537 25988 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.903980 26076 mvcc.cc:205] Tried to move back new op lower bound from 7317213240905338880 to 7317213240687009792. Current Snapshot: MvccSnapshot[applied={T|T < 7317213240905338880}]
I20260811 06:16:53.904861 25986 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.905125 25986 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 7738cda8a42c496bb88cb24f94956ed4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.905704 26075 mvcc.cc:205] Tried to move back new op lower bound from 7317213240905338880 to 7317213240687009792. Current Snapshot: MvccSnapshot[applied={T|T < 7317213240905338880}]
I20260811 06:16:53.906544 26061 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.906808 26061 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:53.908320 25988 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.908602 25988 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 1cf07357005c41909a4cd6a44e826f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:53.911746 26061 tablet_replica.cc:439] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50ba83070b144bae81a498f85b88b085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7738cda8a42c496bb88cb24f94956ed4" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36573 } } peers { permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 } } peers { permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 } } }
I20260811 06:16:53.912050 26061 tablet_replica.cc:442] T 2eeeec11a41946799f1154efad0dc995 P 50ba83070b144bae81a498f85b88b085: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:53.923152 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.916564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.22.121.1:36573)
0811 06:16:53.916758 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.916764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.917217 (+ 453us) server_negotiation.cc:407] Connection header received
0811 06:16:53.917405 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.917408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.917485 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.917600 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:53.918134 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.918886 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.920950 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.921450 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.921544 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.922080 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.922569 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.922674 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.922795 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.922888 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:16:53.941267 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.934499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:54267 (local address 127.22.121.3:41337)
0811 06:16:53.934676 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.934683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.935413 (+ 730us) server_negotiation.cc:407] Connection header received
0811 06:16:53.935550 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.935556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.935637 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.935729 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:53.936318 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.937446 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.939173 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.939997 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.940127 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.940510 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.940640 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.940775 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.940891 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.940974 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:16:53.941488 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.934538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41337 (local address 127.22.121.1:54267)
0811 06:16:53.935319 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.935338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.935445 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.935736 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.935739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.935748 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:53.936168 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.936177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.937584 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.937596 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.939029 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.939041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.940151 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.940156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.940275 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.940325 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.940777 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.940787 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.940853 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.941000 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":438,"threads_started":1}
I20260811 06:16:53.952236 25847 raft_consensus.cc:1261] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Refusing update from remote peer 7738cda8a42c496bb88cb24f94956ed4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.952437 25581 raft_consensus.cc:1261] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Refusing update from remote peer 7738cda8a42c496bb88cb24f94956ed4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:53.953068 26028 consensus_queue.cc:1035] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ba83070b144bae81a498f85b88b085" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 06:16:53.953768 25986 consensus_queue.cc:1035] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cf07357005c41909a4cd6a44e826f3c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:53.997881 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.990765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:59079 (local address 127.22.121.1:36573)
0811 06:16:53.990933 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:53.990940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:53.991890 (+ 950us) server_negotiation.cc:407] Connection header received
0811 06:16:53.992097 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:53.992102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:53.992179 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:53.992250 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:53.992877 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.993833 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.995853 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.996477 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.996591 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.997042 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.997146 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.997219 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:53.997356 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:16:53.997431 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:16:53.997887 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:53.991011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36573 (local address 127.22.121.3:59079)
0811 06:16:53.991801 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:53.991821 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:53.991965 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:53.992286 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:53.992291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:53.992304 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:53.992720 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.992731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.994010 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.994015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.995722 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:53.995738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:53.996666 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:53.996674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:53.996793 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:53.996849 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:53.997241 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:53.997252 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:16:53.997325 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:16:53.997450 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":447,"threads_started":1}
I20260811 06:16:54.036816 25061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:54.037128 25061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:54.040555 25061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c971479a50a14aefa45ef7dfeac4c4a9: Sending DeleteTablet for 3 replicas of tablet 3af5d1fa2e034f879d661a5813672c67
I20260811 06:16:54.041501 25539 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af5d1fa2e034f879d661a5813672c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:54 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38846
I20260811 06:16:54.041812 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af5d1fa2e034f879d661a5813672c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:54 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33120
I20260811 06:16:54.042393 25306 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af5d1fa2e034f879d661a5813672c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:54 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260811 06:16:54.054414 26108 tablet_replica.cc:330] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: stopping tablet replica
I20260811 06:16:54.054836 26108 raft_consensus.cc:2229] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:54.055192 26108 raft_consensus.cc:2258] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:54.056430 26108 ts_tablet_manager.cc:1905] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:54.064595 26108 ts_tablet_manager.cc:1918] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:54.064813 26108 log.cc:1196] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/3af5d1fa2e034f879d661a5813672c67
I20260811 06:16:54.065570 26108 ts_tablet_manager.cc:1939] T 3af5d1fa2e034f879d661a5813672c67 P 50ba83070b144bae81a498f85b88b085: Deleting consensus metadata
I20260811 06:16:54.066720 25041 catalog_manager.cc:4841] TS 50ba83070b144bae81a498f85b88b085 (127.22.121.3:41337): tablet 3af5d1fa2e034f879d661a5813672c67 (table test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]) successfully deleted
I20260811 06:16:54.071755 26112 tablet_replica.cc:330] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: stopping tablet replica
I20260811 06:16:54.072331 26112 raft_consensus.cc:2229] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:16:54.073067 26112 raft_consensus.cc:2258] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:54.077728 26107 tablet_replica.cc:330] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: stopping tablet replica
I20260811 06:16:54.077963 26107 raft_consensus.cc:2229] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:54.078322 26107 raft_consensus.cc:2258] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:54.079417 26107 ts_tablet_manager.cc:1905] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:54.086402 26112 ts_tablet_manager.cc:1905] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:54.087459 26107 ts_tablet_manager.cc:1918] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:54.087646 26107 log.cc:1196] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/3af5d1fa2e034f879d661a5813672c67
I20260811 06:16:54.088155 26107 ts_tablet_manager.cc:1939] T 3af5d1fa2e034f879d661a5813672c67 P 1cf07357005c41909a4cd6a44e826f3c: Deleting consensus metadata
I20260811 06:16:54.089282 25038 catalog_manager.cc:4841] TS 1cf07357005c41909a4cd6a44e826f3c (127.22.121.2:32905): tablet 3af5d1fa2e034f879d661a5813672c67 (table test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]) successfully deleted
I20260811 06:16:54.094130 26112 ts_tablet_manager.cc:1918] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:54.094329 26112 log.cc:1196] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/3af5d1fa2e034f879d661a5813672c67
I20260811 06:16:54.094933 26112 ts_tablet_manager.cc:1939] T 3af5d1fa2e034f879d661a5813672c67 P 7738cda8a42c496bb88cb24f94956ed4: Deleting consensus metadata
I20260811 06:16:54.096333 25038 catalog_manager.cc:4841] TS 7738cda8a42c496bb88cb24f94956ed4 (127.22.121.1:36573): tablet 3af5d1fa2e034f879d661a5813672c67 (table test-table [id=1fcf5734893f4b699e630a5b8e5e06bd]) successfully deleted
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:16:54.108777 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:54.073586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.22.121.1:36573)
0811 06:16:54.073764 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:54.073770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:54.085318 (+ 11548us) server_negotiation.cc:407] Connection header received
0811 06:16:54.085513 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:54.085519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:54.085608 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:54.085728 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:54.091700 (+ 5972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.092956 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.095224 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.095925 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.096042 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:54.101496 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:54.101510 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:54.101528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:54.101573 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:54.105542 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.106408 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.106417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.106421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.106579 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.107006 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.107011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.107019 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.107549 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:54.107899 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:54.108179 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 06:16:54.108466 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 06:16:54.109651 25421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38164
I20260811 06:16:54.131649 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:54.112822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.22.121.62:46701)
0811 06:16:54.113024 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:54.113032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:54.115811 (+ 2779us) server_negotiation.cc:407] Connection header received
0811 06:16:54.115963 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:54.115969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:54.116062 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:54.116161 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:54.117912 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.119095 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.121032 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.121782 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.121954 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:54.124834 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:54.124840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:54.124845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:54.124885 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:54.128392 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.129181 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.129187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.129190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.129340 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.129746 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.129751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.129755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.130097 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:54.130337 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:54.130928 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 06:16:54.131258 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 06:16:54.132251 25061 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37442
I20260811 06:16:54.134521 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25192
I20260811 06:16:54.152915 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:54.136055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.22.121.1:36573)
0811 06:16:54.136249 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:54.136255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:54.137888 (+ 1633us) server_negotiation.cc:407] Connection header received
0811 06:16:54.138058 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:54.138064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:54.138144 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:54.138240 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:54.139662 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.140825 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.142523 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.143214 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.143371 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:54.146150 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:54.146156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:54.146162 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:54.146200 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:54.149731 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.150465 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.150470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.150474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.150650 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.151103 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.151109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.151112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.151481 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:54.151743 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:54.152133 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 06:16:54.152351 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 06:16:54.153498 25421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38178
W20260811 06:16:54.154070 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25192: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:16:54.154260 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25192
I20260811 06:16:54.188414 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25435
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:16:54.203075 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:54.189114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.22.121.2:32905)
0811 06:16:54.189353 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:54.189359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:54.189378 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:16:54.189453 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:54.189458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:54.189535 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:54.189618 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:54.190728 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.191823 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.193048 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.193488 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.193598 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:54.197642 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:54.197653 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:54.197667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:54.197702 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:54.200565 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.201182 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.201189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.201191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.201336 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.201675 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.201679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.201681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.202017 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:54.202305 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:54.202559 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 06:16:54.202771 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 06:16:54.204046 25670 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38854
W20260811 06:16:54.204659 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25435: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:16:54.204797 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25435
I20260811 06:16:54.234488 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25687
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:16:54.250331 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:54.235201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.22.121.3:41337)
0811 06:16:54.235428 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:54.235434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:54.235468 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:16:54.235614 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:54.235620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:54.235703 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:54.235826 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:54.236931 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.238114 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.239410 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:54.240062 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:54.240216 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:54.244574 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:54.244586 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:54.244604 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:54.244642 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:54.247635 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.248428 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.248435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.248437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.248566 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:54.248961 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:54.248964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:54.248966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:54.249310 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:54.249555 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:54.249887 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 06:16:54.250040 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 06:16:54.251142 25932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33142
W20260811 06:16:54.251804 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:16:54.251931 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25687
I20260811 06:16:54.276697 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25012
I20260811 06:16:54.277768 25140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:16:54.387915 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25012
2026-08-11T06:16:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4997 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[26136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[26136](info): set up 2 sockets
Aug 11 06:16:54 dist-test-slave-gjrz krb5kdc[26136](info): commencing operation
krb5kdc: starting...
W20260811 06:16:55.596886 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:16:55 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:16:55.758227 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34159
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:34159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:56.057862 26158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:56.058125 26158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:56.058259 26158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:56.065074 26158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:16:56.065172 26158 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:16:56.065239 26158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:56.065336 26158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:56.065398 26158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:16:56.065479 26158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:16:56.074637 26158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:34159
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:56.075744 26158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:56.076942 26158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:56.088111 26166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:56.088151 26167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:56.088187 26170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:56.088862 26158 server_base.cc:1034] running on GCE node
I20260811 06:16:56.089488 26158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:56.090677 26158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:56.091874 26158 hybrid_clock.cc:648] HybridClock initialized: now 1786429016091814 us; error 37 us; skew 500 ppm
I20260811 06:16:56.092150 26158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:56 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:56.094915 26158 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:16:56.096184 26158 webserver.cc:460] Webserver started at http://127.22.121.62:34775/ using document root <none> and password file <none>
I20260811 06:16:56.096794 26158 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:56.096889 26158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:56.097184 26158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:56.099924 26158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "6bd17551e4d34b1a95104d75488c35db"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "67bd4586e7af9e57a9cc4efe6ffb93db"
server_key_iv: "7619343e8a0deb1e8f67d67688e7c8de"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.100629 26158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "6bd17551e4d34b1a95104d75488c35db"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "67bd4586e7af9e57a9cc4efe6ffb93db"
server_key_iv: "7619343e8a0deb1e8f67d67688e7c8de"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.105293 26158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:16:56.108414 26187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:56.109536 26158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:16:56.109685 26158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "6bd17551e4d34b1a95104d75488c35db"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "67bd4586e7af9e57a9cc4efe6ffb93db"
server_key_iv: "7619343e8a0deb1e8f67d67688e7c8de"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.109862 26158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:56.136523 26158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:56.140060 26158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:56.140295 26158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:56.159606 26158 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:34159
I20260811 06:16:56.159617 26296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:34159 every 8 connection(s)
I20260811 06:16:56.160826 26158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:16:56.164326 26298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:56.170274 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26158
I20260811 06:16:56.170456 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:16:56.171079 23012 external_mini_cluster.cc:1428] Setting key 4d976faccd85b47d83e664d445d1b9f1
I20260811 06:16:56.174438 26298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Bootstrap starting.
I20260811 06:16:56.177919 26298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:56.178910 26298 log.cc:824] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:56.181669 26298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: No bootstrap required, opened a new log
Aug 11 06:16:56 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:56.189522 26298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } }
I20260811 06:16:56.189841 26298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:56.189939 26298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd17551e4d34b1a95104d75488c35db, State: Initialized, Role: FOLLOWER
I20260811 06:16:56.190526 26298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } }
I20260811 06:16:56.190761 26298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:16:56.190927 26298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:16:56.191100 26298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:56.192435 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:56.172859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.22.121.62:34159)
0811 06:16:56.173654 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:56.173665 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:56.173704 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:16:56.175340 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:56.175360 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:56.175732 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:56.176029 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:56.177776 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.179105 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.180614 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.181231 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.181363 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:56.185729 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:56.185737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:56.185750 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:56.185790 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:56.188642 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.189257 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.189263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.189269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.189423 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.189756 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.189760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.189762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.190310 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:56.190618 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:56.190987 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 06:16:56.191510 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":399,"threads_started":1}
I20260811 06:16:56.193336 26298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } }
I20260811 06:16:56.193804 26298 leader_election.cc:302] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bd17551e4d34b1a95104d75488c35db; no voters:
I20260811 06:16:56.194343 26298 leader_election.cc:288] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:16:56.194468 26305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:56.195564 26305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [term 1 LEADER]: Becoming Leader. State: Replica: 6bd17551e4d34b1a95104d75488c35db, State: Running, Role: LEADER
I20260811 06:16:56.196008 26305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } }
I20260811 06:16:56.196348 26298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:16:56.198422 26310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bd17551e4d34b1a95104d75488c35db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } } }
I20260811 06:16:56.198863 26310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:56.198448 26311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bd17551e4d34b1a95104d75488c35db. Latest consensus state: current_term: 1 leader_uuid: "6bd17551e4d34b1a95104d75488c35db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd17551e4d34b1a95104d75488c35db" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34159 } } }
I20260811 06:16:56.201740 26311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db [sys.catalog]: This master's current role is: LEADER
I20260811 06:16:56.204151 26319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:16:56.208743 26319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:16:56.217178 26319 catalog_manager.cc:1347] Generated new cluster ID: af56746e43f64f43b74d1957a2873623
I20260811 06:16:56.217267 26319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:16:56.227990 26319 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:16:56.229230 26319 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:16:56.235953 26319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Generated new TSK 0
I20260811 06:16:56.236575 26319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:16:56.284276 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34159
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:56.589123 26348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:56.589361 26348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:56.589495 26348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:56.596283 26348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:56.596374 26348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:56.596482 26348 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:16:56.596541 26348 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:56.596637 26348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:16:56.605871 26348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:34159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:56.606988 26348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:56.608275 26348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:56.620435 26358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:56.620548 26359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:56.620522 26361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:56.621480 26348 server_base.cc:1034] running on GCE node
I20260811 06:16:56.622043 26348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:56.622867 26348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:56.624064 26348 hybrid_clock.cc:648] HybridClock initialized: now 1786429016624008 us; error 40 us; skew 500 ppm
I20260811 06:16:56.624341 26348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:56 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:56.627144 26348 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:16:56.628490 26348 webserver.cc:460] Webserver started at http://127.22.121.1:46865/ using document root <none> and password file <none>
I20260811 06:16:56.629094 26348 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:56.629208 26348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:56.629487 26348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:56.632193 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "7f48b74ff2cc485e30501a7c883fa0c4"
server_key_iv: "cf10026d5c29828fed299dcd871294c2"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.632884 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "7f48b74ff2cc485e30501a7c883fa0c4"
server_key_iv: "cf10026d5c29828fed299dcd871294c2"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.637576 26348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:16:56.640851 26371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:56.642067 26348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:16:56.642211 26348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
format_stamp: "Formatted at 2026-08-11 06:16:56 on dist-test-slave-gjrz"
server_key: "7f48b74ff2cc485e30501a7c883fa0c4"
server_key_iv: "cf10026d5c29828fed299dcd871294c2"
server_key_version: "encryptionkey@0"
I20260811 06:16:56.642379 26348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:56.658262 26348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:56.661630 26348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:56.661867 26348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:56.664376 26348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:56.664515 26348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:16:56.664621 26348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:56.664712 26348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:16:56 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:56.685849 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:56.664629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42653 (local address 127.22.121.62:34159)
0811 06:16:56.664808 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:56.664814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:56.666030 (+ 1216us) server_negotiation.cc:407] Connection header received
0811 06:16:56.668080 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:56.668086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:56.668175 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:56.668323 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:56.670038 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.671144 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.673416 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.673936 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.674068 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:56.678775 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:56.678779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:56.678782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:56.678813 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:56.681678 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.682701 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.682707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.682710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.682860 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.683376 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.683381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.683384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.683796 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:56.683954 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:56.685298 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0811 06:16:56.685514 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:16:56.687005 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:56.665041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.1:42653)
0811 06:16:56.665852 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:56.665897 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:56.667749 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:56.668597 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:56.668607 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:56.669305 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:56.669867 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:56.669883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.671338 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.671344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:56.673211 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:56.673230 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.674178 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.674186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:56.674489 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:56.675200 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:56.675233 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:56.678573 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:56.681819 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:56.681827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:56.681842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:56.682538 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:56.682960 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:56.682965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:56.682969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:56.683208 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:56.684039 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:56.684047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:56.684546 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 06:16:56.684888 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 06:16:56.685428 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":339,"threads_started":1}
I20260811 06:16:56.704514 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:56.692434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.1:60551)
0811 06:16:56.693005 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:56.693019 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:56.693138 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:56.693410 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:56.693414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:56.693802 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:56.694059 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:56.694067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.695009 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.695012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:56.696406 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:56.696420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.697474 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.697479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:56.697663 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:56.698133 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:56.698156 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:56.699160 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:56.701717 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:56.701723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:56.701728 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:56.702278 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:56.702591 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:56.702596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:56.702600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:56.702747 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:56.703175 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:56.703180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:56.703415 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 06:16:56.703600 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 06:16:56.703962 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":315,"threads_started":1}
I20260811 06:16:56.705202 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:56.692353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:60551 (local address 127.22.121.62:34159)
0811 06:16:56.692502 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:56.692506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:56.693096 (+ 590us) server_negotiation.cc:407] Connection header received
0811 06:16:56.693235 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:56.693240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:56.693303 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:56.693379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:56.694189 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.694872 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.696545 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:56.697112 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:56.697233 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:56.699338 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:56.699344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:56.699349 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:56.699376 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:56.701567 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.702380 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.702384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.702386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.702500 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:56.702831 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:56.702834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:56.702836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:56.703073 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:56.703192 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:56.704619 (+ 1427us) server_negotiation.cc:299] Negotiation successful
0811 06:16:56.704849 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:16:56.709713 26348 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:43481
I20260811 06:16:56.709769 26567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:43481 every 8 connection(s)
I20260811 06:16:56.710762 26348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:16:56.712437 26568 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34159
I20260811 06:16:56.712661 26568 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:56.713215 26568 heartbeater.cc:511] Master 127.22.121.62:34159 requested a full tablet report, sending...
I20260811 06:16:56.714970 26224 ts_manager.cc:194] Registered new tserver with Master: 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481)
I20260811 06:16:56.716446 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:42653
I20260811 06:16:56.719197 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26348
I20260811 06:16:56.719403 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:16:56.720006 23012 external_mini_cluster.cc:1428] Setting key 55629d65d8e662741a7a3056a2158aee
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:16:56.758311 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34159
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:57.063587 26575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:57.063824 26575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:57.063920 26575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:57.070658 26575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:57.070746 26575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:57.070832 26575 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:16:57.070884 26575 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:57.070950 26575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:16:57.079929 26575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:34159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:57.080981 26575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:57.082223 26575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:57.094147 26584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:57.094210 26582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:57.094178 26581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:57.095172 26575 server_base.cc:1034] running on GCE node
I20260811 06:16:57.095736 26575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:57.096593 26575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:57.097792 26575 hybrid_clock.cc:648] HybridClock initialized: now 1786429017097757 us; error 49 us; skew 500 ppm
I20260811 06:16:57.098083 26575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:57.101092 26575 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:16:57.102502 26575 webserver.cc:460] Webserver started at http://127.22.121.2:40065/ using document root <none> and password file <none>
I20260811 06:16:57.103107 26575 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:57.103222 26575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:57.103598 26575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:57.106379 26575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "9a593961d4594e0da229b0efc538c79a"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "7db51fa8f93345d9254f2c3be2b0b652"
server_key_iv: "d9905bf5c02d7365bc2d028a2da3d48d"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.107103 26575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "9a593961d4594e0da229b0efc538c79a"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "7db51fa8f93345d9254f2c3be2b0b652"
server_key_iv: "d9905bf5c02d7365bc2d028a2da3d48d"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.111897 26575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:16:57.115059 26598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:57.116294 26575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:16:57.116441 26575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "9a593961d4594e0da229b0efc538c79a"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "7db51fa8f93345d9254f2c3be2b0b652"
server_key_iv: "d9905bf5c02d7365bc2d028a2da3d48d"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.116627 26575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:57.133617 26575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:57.137454 26575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:57.137705 26575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:57.140233 26575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:57.140332 26575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:57.140463 26575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:57.140563 26575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:57.163923 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.140652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:34113 (local address 127.22.121.62:34159)
0811 06:16:57.140844 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.140850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.142182 (+ 1332us) server_negotiation.cc:407] Connection header received
0811 06:16:57.144300 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.144307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.144412 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.144523 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.146696 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.147844 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.149442 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.150116 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.150243 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.156719 (+ 6476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.156725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.156730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.156772 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.159924 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.160931 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.160937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.160940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.161103 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.161714 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.161719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.161721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.162082 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.162224 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.163402 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.163599 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:16:57.165006 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.141173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.2:34113)
0811 06:16:57.142011 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.142061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.144005 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.144773 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.144784 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.145674 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:57.146493 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.146524 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.148019 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.148023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.149296 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.149313 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.150302 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.150307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.150475 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.153622 (+ 3147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:57.153658 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:57.156498 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:57.160088 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.160097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.160113 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.160755 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.161252 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.161257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.161260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.161545 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.162303 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:57.162311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:57.162776 (+ 465us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.163088 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.163600 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":340,"threads_started":1}
I20260811 06:16:57.185146 26575 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:41713
I20260811 06:16:57.185189 26815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:41713 every 8 connection(s)
I20260811 06:16:57.186311 26575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:16:57.188373 26818 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34159
I20260811 06:16:57.188617 26818 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:57.189491 26818 heartbeater.cc:511] Master 127.22.121.62:34159 requested a full tablet report, sending...
I20260811 06:16:57.190780 26224 ts_manager.cc:194] Registered new tserver with Master: 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713)
I20260811 06:16:57.191701 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:34113
I20260811 06:16:57.192991 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26575
I20260811 06:16:57.193167 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:16:57.193724 23012 external_mini_cluster.cc:1428] Setting key 579f3582d3196ff30f650611c89a9c78
I20260811 06:16:57.196472 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.175487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:54885 (local address 127.22.121.62:34159)
0811 06:16:57.175649 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.175654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.183529 (+ 7875us) server_negotiation.cc:407] Connection header received
0811 06:16:57.183713 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.183717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.183772 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.183857 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.184939 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.185709 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.187013 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.187451 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.187557 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.189764 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.189771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.189776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.189818 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.193304 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.194099 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.194104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.194107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.194258 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.195137 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.195140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.195143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.195530 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.195661 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.196074 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.196225 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 06:16:57.196674 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.175293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.2:54885)
0811 06:16:57.183442 (+ 8149us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.183460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.183606 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.183905 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.183910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.184475 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:57.184809 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.184819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.185841 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.185846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.186881 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.186896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.187630 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.187637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.187804 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.188446 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:57.188468 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:57.189520 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:57.193469 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.193474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.193478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.193963 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.194410 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.194414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.194417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.194576 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.195651 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:57.195655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:57.195847 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.196005 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.196304 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8000,"thread_start_us":7804,"threads_started":1}
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:16:57.233549 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34159
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:16:57.536196 26826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:16:57.536432 26826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:16:57.536530 26826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:16:57.543285 26826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:16:57.543371 26826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:16:57.543457 26826 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:16:57.543506 26826 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:16:57.543571 26826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:16:57.552531 26826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:34159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:16:57.553557 26826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:16:57.554703 26826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:16:57.566692 26839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:57.566663 26840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:16:57.566642 26843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:16:57.567853 26826 server_base.cc:1034] running on GCE node
I20260811 06:16:57.568480 26826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:16:57.569272 26826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:16:57.570461 26826 hybrid_clock.cc:648] HybridClock initialized: now 1786429017570425 us; error 28 us; skew 500 ppm
I20260811 06:16:57.570709 26826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:16:57.573510 26826 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:16:57.574811 26826 webserver.cc:460] Webserver started at http://127.22.121.3:42545/ using document root <none> and password file <none>
I20260811 06:16:57.575453 26826 fs_manager.cc:359] Metadata directory not provided
I20260811 06:16:57.575578 26826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:16:57.575870 26826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:16:57.578647 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "0de7719e897b0387c099be54e055f201"
server_key_iv: "bd436912d8cd3c1c4e4260e612d45e26"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.579401 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "0de7719e897b0387c099be54e055f201"
server_key_iv: "bd436912d8cd3c1c4e4260e612d45e26"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.584132 26826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:16:57.587203 26852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:57.588418 26826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:16:57.588559 26826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
format_stamp: "Formatted at 2026-08-11 06:16:57 on dist-test-slave-gjrz"
server_key: "0de7719e897b0387c099be54e055f201"
server_key_iv: "bd436912d8cd3c1c4e4260e612d45e26"
server_key_version: "encryptionkey@0"
I20260811 06:16:57.588721 26826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:16:57.606698 26826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:16:57.610045 26826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:16:57.610280 26826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:16:57.612764 26826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:16:57.612896 26826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:16:57.613077 26826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:16:57.613194 26826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:57.637854 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.613694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.3:43899)
0811 06:16:57.614569 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.614613 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.616602 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.617348 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.617358 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.618182 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:57.619211 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.619243 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.620815 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.620820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.623320 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.623339 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.624308 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.624315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.624628 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.625333 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:57.625372 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:57.628806 (+ 3434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:57.632632 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.632640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.632656 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.633222 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.633700 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.633706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.633710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.634029 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.634792 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:57.634801 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:57.635332 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.635664 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.636229 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":360,"threads_started":1}
I20260811 06:16:57.639173 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.613117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:43899 (local address 127.22.121.62:34159)
0811 06:16:57.613319 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.613325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.614726 (+ 1401us) server_negotiation.cc:407] Connection header received
0811 06:16:57.616871 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.616877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.616969 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.617096 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.619481 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.620612 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.623496 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.624064 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.624168 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.629091 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.629098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.629103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.629144 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.632459 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.633364 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.633369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.633373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.633551 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.634197 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.634202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.634205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.634563 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.634710 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.638588 (+ 3878us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.638828 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:16:57.649936 26826 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:35701
I20260811 06:16:57.650029 27074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:35701 every 8 connection(s)
I20260811 06:16:57.651367 26826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:16:57.653394 27076 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34159
I20260811 06:16:57.653620 27076 heartbeater.cc:464] Registering TS with master...
I20260811 06:16:57.654206 27076 heartbeater.cc:511] Master 127.22.121.62:34159 requested a full tablet report, sending...
I20260811 06:16:57.655545 26224 ts_manager.cc:194] Registered new tserver with Master: 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:16:57.656427 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26826
I20260811 06:16:57.656507 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:43899
I20260811 06:16:57.656615 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:16:57.657166 23012 external_mini_cluster.cc:1428] Setting key 27cd5bb4a35129adeab3947eca7fd82b
I20260811 06:16:57.659936 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:16:57.661567 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.649596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.22.121.3:58213)
0811 06:16:57.650296 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.650314 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.650414 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.650703 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.650706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.651094 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:57.651414 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.651427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.652652 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.652655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.653804 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.653817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.654556 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.654563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.654723 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.655209 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:57.655229 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:57.655921 (+ 692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:57.659393 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.659398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.659401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.659885 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.660271 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.660274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.660276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.660390 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.660860 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:57.660863 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:57.660996 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.661100 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.661291 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":350,"threads_started":1}
I20260811 06:16:57.661731 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.649500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58213 (local address 127.22.121.62:34159)
0811 06:16:57.649658 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.649664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.650391 (+ 727us) server_negotiation.cc:407] Connection header received
0811 06:16:57.650494 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.650500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.650583 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.650684 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.651543 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.652519 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.653946 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.654375 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.654488 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.656110 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.656114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.656117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.656146 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.659252 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.659997 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.660003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.660006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.660172 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.660476 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.660480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.660483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.660768 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.660906 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.661197 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.661393 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:16:57.704135 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.691975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.22.121.62:34159)
0811 06:16:57.692107 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.692112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.692493 (+ 381us) server_negotiation.cc:407] Connection header received
0811 06:16:57.692624 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.692627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.692693 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.692782 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.694012 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.694724 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.695894 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.696294 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.696399 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.699850 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.699856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.699860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.699900 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.702160 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.702682 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.702686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.702688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.702785 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.703042 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.703045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.703047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.703343 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.703514 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.703767 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.703915 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:16:57.707713 26224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:16:57.714457 26224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:57.718736 26568 heartbeater.cc:503] Master 127.22.121.62:34159 was elected leader, sending a full tablet report...
I20260811 06:16:57.734463 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.726335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.22.121.2:41713)
0811 06:16:57.727094 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.727102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.727130 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:16:57.727239 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.727246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.727514 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.727704 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:57.728417 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.729955 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.732474 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.733334 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.733458 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.733667 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.734042 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.734206 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":1367,"threads_started":1}
I20260811 06:16:57.734539 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.726002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:41713 (local address 127.0.0.1:35768)
0811 06:16:57.726725 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.726793 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.726955 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.727725 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.727731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.727762 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:57.728190 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.728203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.730099 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.730104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.732298 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.732316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.733549 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.733555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.733812 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.733873 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.733982 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.734146 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":155,"thread_start_us":329,"threads_started":1}
I20260811 06:16:57.735229 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.725453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.0.0.1:42242)
0811 06:16:57.726726 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.727119 (+ 393us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.727303 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.728070 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.728075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.728099 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:57.728531 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.728545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.730149 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.730159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.732329 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.732344 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.734436 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.734442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.734623 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.734656 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.734750 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.734876 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":61,"thread_start_us":1025,"threads_started":1}
I20260811 06:16:57.735338 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.725562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.22.121.1:43481)
0811 06:16:57.726163 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.726172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.727189 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 06:16:57.727480 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.727487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.727684 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.727824 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:57.728739 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.729964 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.732527 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.733963 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.734096 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.734288 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.734867 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.735048 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":274,"threads_started":1}
I20260811 06:16:57.737391 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.727059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35701 (local address 127.0.0.1:42760)
0811 06:16:57.728814 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.728831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.728955 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.729479 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.729484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.729507 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:57.729816 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.729826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.731788 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.731793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.734881 (+ 3088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.734894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.736646 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.736652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.736820 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.736851 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.736954 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.737072 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1646,"mutex_wait_us":22,"thread_start_us":1502,"threads_started":1}
I20260811 06:16:57.737461 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.727560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.22.121.3:35701)
0811 06:16:57.728381 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.728389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.728931 (+ 542us) server_negotiation.cc:407] Connection header received
0811 06:16:57.729090 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.729097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.729301 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.729471 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:57.730051 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.731623 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.735050 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.736431 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.736570 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.736785 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.736996 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.737158 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":506,"threads_started":1}
I20260811 06:16:57.738600 26464 tablet_service.cc:1467] Processing CreateTablet for tablet ce64d8548b3d4bc28a6ada377e04099e (DEFAULT_TABLE table=test-table [id=a9400b89e3494d838e0d1b3a73421c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:57.738634 26691 tablet_service.cc:1467] Processing CreateTablet for tablet ce64d8548b3d4bc28a6ada377e04099e (DEFAULT_TABLE table=test-table [id=a9400b89e3494d838e0d1b3a73421c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:57.739833 26464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce64d8548b3d4bc28a6ada377e04099e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:57.739928 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce64d8548b3d4bc28a6ada377e04099e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:57.742707 26956 tablet_service.cc:1467] Processing CreateTablet for tablet ce64d8548b3d4bc28a6ada377e04099e (DEFAULT_TABLE table=test-table [id=a9400b89e3494d838e0d1b3a73421c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:57.744670 26956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce64d8548b3d4bc28a6ada377e04099e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:57.750834 27109 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Bootstrap starting.
I20260811 06:16:57.753754 27109 tablet_bootstrap.cc:654] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.754487 27110 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Bootstrap starting.
I20260811 06:16:57.755033 27109 log.cc:824] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:57.757591 27110 tablet_bootstrap.cc:654] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.757995 27109 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: No bootstrap required, opened a new log
I20260811 06:16:57.758291 27109 ts_tablet_manager.cc:1397] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 06:16:57.758715 27112 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Bootstrap starting.
I20260811 06:16:57.758765 27110 log.cc:824] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:57.761819 27110 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: No bootstrap required, opened a new log
I20260811 06:16:57.761900 27112 tablet_bootstrap.cc:654] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.762094 27110 ts_tablet_manager.cc:1397] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260811 06:16:57.762856 27112 log.cc:824] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Log is configured to *not* fsync() on all Append() calls
I20260811 06:16:57.765122 27112 tablet_bootstrap.cc:492] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: No bootstrap required, opened a new log
I20260811 06:16:57.765432 27112 ts_tablet_manager.cc:1397] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 06:16:57.767908 27109 raft_consensus.cc:348] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.768313 27109 raft_consensus.cc:374] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.768405 27109 raft_consensus.cc:729] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.769053 27109 consensus_queue.cc:260] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.770648 27109 ts_tablet_manager.cc:1428] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 06:16:57.771777 27110 raft_consensus.cc:348] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.772068 27110 raft_consensus.cc:374] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.772157 27110 raft_consensus.cc:729] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.772660 27110 consensus_queue.cc:260] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.773531 26818 heartbeater.cc:503] Master 127.22.121.62:34159 was elected leader, sending a full tablet report...
I20260811 06:16:57.774180 27110 ts_tablet_manager.cc:1428] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260811 06:16:57.773936 27112 raft_consensus.cc:348] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.774309 27112 raft_consensus.cc:374] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.774405 27112 raft_consensus.cc:729] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.775170 27112 consensus_queue.cc:260] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.779873 27076 heartbeater.cc:503] Master 127.22.121.62:34159 was elected leader, sending a full tablet report...
I20260811 06:16:57.780354 27112 ts_tablet_manager.cc:1428] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 06:16:57.878072 27117 raft_consensus.cc:482] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:57.878328 27117 raft_consensus.cc:504] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.879892 27117 leader_election.cc:288] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:16:57.885859 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.880277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:53699 (local address 127.22.121.1:43481)
0811 06:16:57.880489 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.880496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.880914 (+ 418us) server_negotiation.cc:407] Connection header received
0811 06:16:57.881087 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.881093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.881186 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.881279 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:57.881851 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.882549 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.884282 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.885028 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.885163 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.885236 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.885511 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.885641 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:16:57.885941 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.880200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.22.121.2:53699)
0811 06:16:57.880786 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.880802 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.880939 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.881395 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.881400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.881432 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:57.881722 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.881731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.882682 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.882686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.884142 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.884154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.885232 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.885239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.885371 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.885394 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.885474 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.885593 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":314,"threads_started":1}
I20260811 06:16:57.886860 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.881123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:42541 (local address 127.22.121.3:35701)
0811 06:16:57.881317 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.881322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.881590 (+ 268us) server_negotiation.cc:407] Connection header received
0811 06:16:57.881749 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.881753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.881809 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.881882 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:57.882421 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.883114 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.884800 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.885530 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.885685 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.885751 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.886503 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.886631 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:16:57.886930 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.880867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35701 (local address 127.22.121.2:42541)
0811 06:16:57.881483 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.881496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.881616 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.881923 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.881927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.881949 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:57.882275 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.882285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.883255 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.883281 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.884648 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.884660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.885745 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.885751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.885917 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.886044 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.886459 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.886582 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":1,"thread_start_us":317,"threads_started":1}
I20260811 06:16:57.886785 26498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce64d8548b3d4bc28a6ada377e04099e" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" is_pre_election: true
I20260811 06:16:57.887291 26498 raft_consensus.cc:2454] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 0.
I20260811 06:16:57.887794 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce64d8548b3d4bc28a6ada377e04099e" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" is_pre_election: true
I20260811 06:16:57.887910 26607 leader_election.cc:302] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:16:57.888290 27117 raft_consensus.cc:2781] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:57.888314 26984 raft_consensus.cc:2454] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 0.
I20260811 06:16:57.888408 27117 raft_consensus.cc:482] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:57.888500 27117 raft_consensus.cc:3037] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:57.890913 27117 raft_consensus.cc:504] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.891505 27117 leader_election.cc:288] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 election: Requested vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:16:57.891816 26498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce64d8548b3d4bc28a6ada377e04099e" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
I20260811 06:16:57.891986 26498 raft_consensus.cc:3037] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:57.891963 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce64d8548b3d4bc28a6ada377e04099e" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
I20260811 06:16:57.892151 26984 raft_consensus.cc:3037] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:57.894132 26498 raft_consensus.cc:2454] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 1.
I20260811 06:16:57.894431 26984 raft_consensus.cc:2454] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 1.
I20260811 06:16:57.894431 26607 leader_election.cc:302] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:16:57.894683 27117 raft_consensus.cc:2781] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:57.895117 27117 raft_consensus.cc:686] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 1 LEADER]: Becoming Leader. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Running, Role: LEADER
I20260811 06:16:57.895663 27117 consensus_queue.cc:237] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } }
I20260811 06:16:57.899919 26222 catalog_manager.cc:5495] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a593961d4594e0da229b0efc538c79a (127.22.121.2). New cstate: current_term: 1 leader_uuid: "9a593961d4594e0da229b0efc538c79a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } health_report { overall_health: HEALTHY } } }
W20260811 06:16:57.902743 27077 tablet.cc:2369] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:16:57 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
W20260811 06:16:57.944386 26819 tablet.cc:2369] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:16:57.946444 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.933301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34159 (local address 127.0.0.1:43058)
0811 06:16:57.933844 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:57.933859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:57.933965 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:57.934223 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:57.934226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:57.934827 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:16:57.935090 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.935100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.936086 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.936090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.937149 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:57.937161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.938105 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.938112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:57.938295 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.938989 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:16:57.939020 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:16:57.941443 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:16:57.943983 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.943995 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.944002 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.944535 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.944883 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:16:57.944887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:16:57.944890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:16:57.945066 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:16:57.945575 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:16:57.945587 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:16:57.945752 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 06:16:57.945878 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 06:16:57.946061 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":300,"threads_started":1}
I20260811 06:16:57.946493 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:57.933249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.22.121.62:34159)
0811 06:16:57.933423 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:57.933429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:57.933939 (+ 510us) server_negotiation.cc:407] Connection header received
0811 06:16:57.934049 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:57.934052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:57.934112 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:57.934185 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:16:57.935219 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.935977 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.937295 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:57.937936 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:57.938033 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:57.941607 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:16:57.941612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:16:57.941615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:16:57.941642 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:16:57.943868 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.944647 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.944652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.944654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.944774 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:16:57.945169 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:16:57.945172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:16:57.945174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:16:57.945438 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:16:57.945573 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:57.945959 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 06:16:57.946219 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":56}
I20260811 06:16:57.950896 26222 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:43058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:16:57.951874 26222 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:57.957551 26464 tablet_service.cc:1467] Processing CreateTablet for tablet c71f384b058a4e0383dadb6da998d303 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f65691c3344645e78b1edae36eed4cc3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:57.957785 26691 tablet_service.cc:1467] Processing CreateTablet for tablet c71f384b058a4e0383dadb6da998d303 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f65691c3344645e78b1edae36eed4cc3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:57.958475 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71f384b058a4e0383dadb6da998d303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:57.958511 26464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71f384b058a4e0383dadb6da998d303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:57.958066 26956 tablet_service.cc:1467] Processing CreateTablet for tablet c71f384b058a4e0383dadb6da998d303 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f65691c3344645e78b1edae36eed4cc3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:16:57.958860 26956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71f384b058a4e0383dadb6da998d303. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 06:16:57.962888 26569 tablet.cc:2369] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:16:57.964438 27110 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: Bootstrap starting.
I20260811 06:16:57.965459 27109 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: Bootstrap starting.
I20260811 06:16:57.965896 27110 tablet_bootstrap.cc:654] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.966988 27109 tablet_bootstrap.cc:654] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.967538 27112 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: Bootstrap starting.
I20260811 06:16:57.968257 27110 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: No bootstrap required, opened a new log
I20260811 06:16:57.968389 27110 ts_tablet_manager.cc:1397] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 06:16:57.969097 27112 tablet_bootstrap.cc:654] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:57.969121 27110 raft_consensus.cc:348] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.969364 27110 raft_consensus.cc:374] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.969501 27109 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: No bootstrap required, opened a new log
I20260811 06:16:57.969491 27110 raft_consensus.cc:729] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.969635 27109 ts_tablet_manager.cc:1397] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:16:57.969753 27110 consensus_queue.cc:260] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.970358 27110 ts_tablet_manager.cc:1428] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 06:16:57.970229 27117 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:57.971197 27117 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:57.971544 27112 tablet_bootstrap.cc:492] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: No bootstrap required, opened a new log
I20260811 06:16:57.971695 27112 ts_tablet_manager.cc:1397] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:16:57.971735 27109 raft_consensus.cc:348] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.971944 27109 raft_consensus.cc:374] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.972044 27109 raft_consensus.cc:729] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.972266 27109 consensus_queue.cc:260] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.972401 27112 raft_consensus.cc:348] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.972641 27112 raft_consensus.cc:374] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:57.972738 27112 raft_consensus.cc:729] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Initialized, Role: FOLLOWER
I20260811 06:16:57.972707 27116 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:57.973037 27116 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:57.972957 27112 consensus_queue.cc:260] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:57.973481 27112 ts_tablet_manager.cc:1428] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:16:57.973372 27123 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:57.973716 27123 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:57.973977 27109 ts_tablet_manager.cc:1428] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:16:58.273239 27116 raft_consensus.cc:482] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:58.273451 27116 raft_consensus.cc:504] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.274912 27116 leader_election.cc:288] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:16:58.282752 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.275251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:38571 (local address 127.22.121.3:35701)
0811 06:16:58.275472 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.275479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.276556 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 06:16:58.276788 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.276791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.276849 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.276915 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:58.277416 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.278351 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.280640 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.281752 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.281917 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.281993 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.282361 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.282490 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:16:58.282790 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.275246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:47611 (local address 127.22.121.2:41713)
0811 06:16:58.276210 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.276216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.276235 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:16:58.276309 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.276313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.276388 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.276473 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:58.277075 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.278060 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.280642 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.281835 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.281993 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.282057 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.282360 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.282492 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":695,"threads_started":1}
I20260811 06:16:58.282943 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.275248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:41713 (local address 127.22.121.1:47611)
0811 06:16:58.275912 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.275934 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.276068 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.276527 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.276532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.276561 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:58.276894 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.276907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.278198 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.278201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.280486 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.280502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.282018 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.282024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.282179 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.282255 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.282329 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.282468 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":98,"thread_start_us":359,"threads_started":1}
I20260811 06:16:58.283042 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.275471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35701 (local address 127.22.121.1:38571)
0811 06:16:58.276443 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.276459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.276663 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.276959 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.276963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.276988 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:58.277274 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.277281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.278468 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.278472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.280502 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.280513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.281964 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.281970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.282163 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.282229 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.282329 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.282468 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":36,"spinlock_wait_cycles":26880,"thread_start_us":709,"threads_started":1}
I20260811 06:16:58.283635 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71f384b058a4e0383dadb6da998d303" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" is_pre_election: true
I20260811 06:16:58.283855 26984 raft_consensus.cc:2454] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 0.
I20260811 06:16:58.283829 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71f384b058a4e0383dadb6da998d303" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a" is_pre_election: true
I20260811 06:16:58.284206 26721 raft_consensus.cc:2454] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 0.
I20260811 06:16:58.284272 26381 leader_election.cc:302] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 5a0b479ad79a43259b3a7c17ca09f4df; no voters:
I20260811 06:16:58.284654 27116 raft_consensus.cc:2781] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:58.284787 27116 raft_consensus.cc:482] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:58.284890 27116 raft_consensus.cc:3037] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.287062 27116 raft_consensus.cc:504] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.287621 27116 leader_election.cc:288] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 election: Requested vote from peers 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:16:58.288004 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71f384b058a4e0383dadb6da998d303" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a"
I20260811 06:16:58.288053 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71f384b058a4e0383dadb6da998d303" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
I20260811 06:16:58.288177 26721 raft_consensus.cc:3037] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.288224 26984 raft_consensus.cc:3037] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.290324 26984 raft_consensus.cc:2454] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 1.
I20260811 06:16:58.290355 26721 raft_consensus.cc:2454] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 1.
I20260811 06:16:58.290737 26374 leader_election.cc:302] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:16:58.290987 27116 raft_consensus.cc:2781] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:58.291380 27116 raft_consensus.cc:686] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 LEADER]: Becoming Leader. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Running, Role: LEADER
I20260811 06:16:58.291853 27116 consensus_queue.cc:237] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.294862 27150 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: TxnStatusTablet state changed. Reason: New leader 5537732d10ba4a248b14e6ad5b8f5b25. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.295254 27150 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:58.295950 27158 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:16:58.296227 27158 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:16:58.296176 26219 catalog_manager.cc:5495] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260811 06:16:58.337306 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.331466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.22.121.1:43481)
0811 06:16:58.331653 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.331657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.331702 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:16:58.331978 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.331982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.332053 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.332172 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.332558 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.333336 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.335187 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.335677 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.335769 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.336294 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.336769 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.336877 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.337006 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.337072 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:16:58.337442 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.331369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.0.0.1:42254)
0811 06:16:58.331627 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.331648 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.331796 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.332220 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.332225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.332238 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:58.332441 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.332451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.333461 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.333467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.335059 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.335074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.335835 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.335841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.336059 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.336131 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.336872 (+ 741us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.336893 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.336982 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.337145 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:16:58.345337 26984 raft_consensus.cc:1261] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Refusing update from remote peer 5537732d10ba4a248b14e6ad5b8f5b25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.345315 26721 raft_consensus.cc:1261] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Refusing update from remote peer 5537732d10ba4a248b14e6ad5b8f5b25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.346241 27116 consensus_queue.cc:1035] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.346525 27150 consensus_queue.cc:1035] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.355886 27117 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: TxnStatusTablet state changed. Reason: New leader 5537732d10ba4a248b14e6ad5b8f5b25. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.356220 27117 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:58.357957 27165 mvcc.cc:205] Tried to move back new op lower bound from 7317213259133231104 to 7317213258926800896. Current Snapshot: MvccSnapshot[applied={T|T < 7317213259133231104}]
I20260811 06:16:58.358820 27116 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.359073 27116 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:58.359468 27169 mvcc.cc:205] Tried to move back new op lower bound from 7317213259133231104 to 7317213258926800896. Current Snapshot: MvccSnapshot[applied={T|T < 7317213259133231104}]
I20260811 06:16:58.360415 27117 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.360671 27117 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 9a593961d4594e0da229b0efc538c79a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:58.366008 27150 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.366247 27150 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5537732d10ba4a248b14e6ad5b8f5b25: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:16:58.366448 27123 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: TxnStatusTablet state changed. Reason: New leader 5537732d10ba4a248b14e6ad5b8f5b25. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.366700 27123 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:58.369148 27123 tablet_replica.cc:439] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } } }
I20260811 06:16:58.369346 27123 tablet_replica.cc:442] T c71f384b058a4e0383dadb6da998d303 P 5a0b479ad79a43259b3a7c17ca09f4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:16:58.382297 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.376254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35778 (local address 127.22.121.2:41713)
0811 06:16:58.376451 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.376458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.376927 (+ 469us) server_negotiation.cc:407] Connection header received
0811 06:16:58.377124 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.377127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.377207 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.377317 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.377762 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.378457 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.380409 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.380785 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.380872 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.381325 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.381764 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.381861 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.381993 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.382053 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:16:58.396742 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.391318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55803 (local address 127.22.121.1:43481)
0811 06:16:58.391457 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.391463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.391895 (+ 432us) server_negotiation.cc:407] Connection header received
0811 06:16:58.392046 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.392052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.392135 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.392209 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.392606 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.393465 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.394827 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.395309 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.395436 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.396054 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.396159 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.396229 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.396368 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.396444 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 06:16:58.396832 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.391234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.22.121.2:55803)
0811 06:16:58.391778 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.391792 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.391927 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.392238 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.392243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.392254 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:58.392448 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.392459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.393605 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.393610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.394695 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.394708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.395709 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.395714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.395830 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.395877 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.396252 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.396264 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.396332 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.396460 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":249,"threads_started":1}
I20260811 06:16:58.405893 26984 raft_consensus.cc:1261] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Refusing update from remote peer 9a593961d4594e0da229b0efc538c79a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.406208 26498 raft_consensus.cc:1261] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Refusing update from remote peer 9a593961d4594e0da229b0efc538c79a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.406894 27117 consensus_queue.cc:1035] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.407387 27189 consensus_queue.cc:1035] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.447141 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.440316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:50243 (local address 127.22.121.2:41713)
0811 06:16:58.440482 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.440488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.441225 (+ 737us) server_negotiation.cc:407] Connection header received
0811 06:16:58.441401 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.441406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.441485 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.441573 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.442270 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.443510 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.445236 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.445907 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.446032 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.446467 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.446603 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.446687 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.446791 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.446866 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:16:58.447207 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.440377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:41713 (local address 127.22.121.1:50243)
0811 06:16:58.441098 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.441118 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.441269 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.441609 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.441614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.441625 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:58.442094 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.442110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.443637 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.443641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.445052 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.445066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.446125 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.446130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.446250 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.446300 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.446687 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.446695 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.446767 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.446894 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":365,"threads_started":1}
I20260811 06:16:58.482389 26219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:58.482671 26219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:16:58.486317 26219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Sending DeleteTablet for 3 replicas of tablet ce64d8548b3d4bc28a6ada377e04099e
I20260811 06:16:58.487429 26956 tablet_service.cc:1514] Processing DeleteTablet for tablet ce64d8548b3d4bc28a6ada377e04099e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42760
I20260811 06:16:58.487607 26464 tablet_service.cc:1514] Processing DeleteTablet for tablet ce64d8548b3d4bc28a6ada377e04099e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42242
I20260811 06:16:58.488677 26691 tablet_service.cc:1514] Processing DeleteTablet for tablet ce64d8548b3d4bc28a6ada377e04099e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:16:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:35768
I20260811 06:16:58.495559 26219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:16:58.497512 26219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:16:58.499874 27205 tablet_replica.cc:330] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: stopping tablet replica
I20260811 06:16:58.500396 27205 raft_consensus.cc:2229] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:58.500519 27207 tablet_replica.cc:330] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: stopping tablet replica
I20260811 06:16:58.500771 27205 raft_consensus.cc:2258] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:58.500968 27207 raft_consensus.cc:2229] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:16:58.501713 27207 raft_consensus.cc:2258] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:58.502563 27205 ts_tablet_manager.cc:1905] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:58.503454 27207 ts_tablet_manager.cc:1905] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:58.506471 26956 tablet_service.cc:1467] Processing CreateTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 (DEFAULT_TABLE table=test-table [id=4a5e054b7af14844915bb9c3bc8ab373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:58.506824 26460 tablet_service.cc:1467] Processing CreateTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 (DEFAULT_TABLE table=test-table [id=4a5e054b7af14844915bb9c3bc8ab373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:58.507042 26956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad9b94933d834f9e8c9078b84cd5ecf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:58.507148 26691 tablet_service.cc:1467] Processing CreateTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 (DEFAULT_TABLE table=test-table [id=4a5e054b7af14844915bb9c3bc8ab373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:16:58.507293 26460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad9b94933d834f9e8c9078b84cd5ecf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:58.507532 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad9b94933d834f9e8c9078b84cd5ecf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:16:58.518805 27205 ts_tablet_manager.cc:1918] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:58.519512 27205 log.cc:1196] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/ce64d8548b3d4bc28a6ada377e04099e
I20260811 06:16:58.520716 27205 ts_tablet_manager.cc:1939] T ce64d8548b3d4bc28a6ada377e04099e P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting consensus metadata
I20260811 06:16:58.523061 26196 catalog_manager.cc:4841] TS 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701): tablet ce64d8548b3d4bc28a6ada377e04099e (table test-table [id=a9400b89e3494d838e0d1b3a73421c07]) successfully deleted
I20260811 06:16:58.524377 27206 tablet_replica.cc:330] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: stopping tablet replica
I20260811 06:16:58.525602 27208 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Bootstrap starting.
I20260811 06:16:58.525868 27206 raft_consensus.cc:2229] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:16:58.526273 27207 ts_tablet_manager.cc:1918] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:58.526304 27206 raft_consensus.cc:2258] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:16:58.527745 27208 tablet_bootstrap.cc:654] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:58.529601 27206 ts_tablet_manager.cc:1905] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:16:58.531548 27207 log.cc:1196] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/ce64d8548b3d4bc28a6ada377e04099e
I20260811 06:16:58.532141 27207 ts_tablet_manager.cc:1939] T ce64d8548b3d4bc28a6ada377e04099e P 9a593961d4594e0da229b0efc538c79a: Deleting consensus metadata
I20260811 06:16:58.532650 27209 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Bootstrap starting.
I20260811 06:16:58.526798 27210 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Bootstrap starting.
I20260811 06:16:58.534385 26189 catalog_manager.cc:4841] TS 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713): tablet ce64d8548b3d4bc28a6ada377e04099e (table test-table [id=a9400b89e3494d838e0d1b3a73421c07]) successfully deleted
I20260811 06:16:58.535919 27208 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: No bootstrap required, opened a new log
I20260811 06:16:58.535965 27210 tablet_bootstrap.cc:654] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:58.536041 27209 tablet_bootstrap.cc:654] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Neither blocks nor log segments found. Creating new log.
I20260811 06:16:58.536108 27208 ts_tablet_manager.cc:1397] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260811 06:16:58.536831 27208 raft_consensus.cc:348] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.537333 27208 raft_consensus.cc:374] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:58.537462 27208 raft_consensus.cc:729] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Initialized, Role: FOLLOWER
I20260811 06:16:58.537744 27208 consensus_queue.cc:260] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.540524 27208 ts_tablet_manager.cc:1428] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 06:16:58.543918 27210 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: No bootstrap required, opened a new log
I20260811 06:16:58.544114 27210 ts_tablet_manager.cc:1397] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260811 06:16:58.544910 27210 raft_consensus.cc:348] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.545202 27210 raft_consensus.cc:374] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:58.545341 27210 raft_consensus.cc:729] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Initialized, Role: FOLLOWER
I20260811 06:16:58.545711 27210 consensus_queue.cc:260] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.549470 27210 ts_tablet_manager.cc:1428] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 06:16:58.549860 27206 ts_tablet_manager.cc:1918] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:16:58.550033 27206 log.cc:1196] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/ce64d8548b3d4bc28a6ada377e04099e
I20260811 06:16:58.550601 27206 ts_tablet_manager.cc:1939] T ce64d8548b3d4bc28a6ada377e04099e P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting consensus metadata
I20260811 06:16:58.552430 26196 catalog_manager.cc:4841] TS 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481): tablet ce64d8548b3d4bc28a6ada377e04099e (table test-table [id=a9400b89e3494d838e0d1b3a73421c07]) successfully deleted
I20260811 06:16:58.553972 27209 tablet_bootstrap.cc:492] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: No bootstrap required, opened a new log
I20260811 06:16:58.554131 27209 ts_tablet_manager.cc:1397] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.005s
I20260811 06:16:58.554888 27209 raft_consensus.cc:348] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.555133 27209 raft_consensus.cc:374] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:16:58.555233 27209 raft_consensus.cc:729] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Initialized, Role: FOLLOWER
I20260811 06:16:58.555521 27209 consensus_queue.cc:260] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.557046 27209 ts_tablet_manager.cc:1428] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:16:58.666934 27123 raft_consensus.cc:482] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:16:58.667248 27123 raft_consensus.cc:504] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.669396 27123 leader_election.cc:288] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713)
I20260811 06:16:58.691426 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.670377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:36771 (local address 127.22.121.1:43481)
0811 06:16:58.670608 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.670615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.683764 (+ 13149us) server_negotiation.cc:407] Connection header received
0811 06:16:58.684047 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.684054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.684132 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.684199 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:58.684978 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.686147 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.688776 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.690122 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.690255 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.690325 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.690988 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.691122 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:16:58.692133 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.669944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.22.121.3:36771)
0811 06:16:58.683662 (+ 13718us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.683681 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.683879 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.684402 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.684407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.684433 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:58.684800 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.684812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.686330 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.686335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.688535 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.688551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.690587 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.690593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.690784 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.690818 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.691655 (+ 837us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.691785 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13583,"mutex_wait_us":1,"thread_start_us":16422,"threads_started":1}
I20260811 06:16:58.693785 26498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad9b94933d834f9e8c9078b84cd5ecf6" candidate_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" is_pre_election: true
I20260811 06:16:58.694048 26498 raft_consensus.cc:2454] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a0b479ad79a43259b3a7c17ca09f4df in term 0.
I20260811 06:16:58.694700 26857 leader_election.cc:302] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 5a0b479ad79a43259b3a7c17ca09f4df; no voters:
I20260811 06:16:58.695133 27123 raft_consensus.cc:2781] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:16:58.695250 27123 raft_consensus.cc:482] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:16:58.695412 27123 raft_consensus.cc:3037] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.697096 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.670838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58331 (local address 127.22.121.2:41713)
0811 06:16:58.671021 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.671027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.686355 (+ 15328us) server_negotiation.cc:407] Connection header received
0811 06:16:58.686764 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.686769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.686839 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.686893 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:16:58.687732 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.688847 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.691323 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.692670 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.692807 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.692897 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.696704 (+ 3807us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.696839 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:16:58.697885 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.670632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:41713 (local address 127.22.121.3:58331)
0811 06:16:58.686247 (+ 15615us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.686264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.686592 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.687059 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.687064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.687087 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:16:58.687542 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.687553 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.689037 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.689042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.691116 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.691132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.695921 (+ 4789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.695928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.696505 (+ 577us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.696546 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.697307 (+ 761us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.697442 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15503,"mutex_wait_us":1,"thread_start_us":12755,"threads_started":1}
I20260811 06:16:58.698551 27123 raft_consensus.cc:504] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.698628 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad9b94933d834f9e8c9078b84cd5ecf6" candidate_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a" is_pre_election: true
I20260811 06:16:58.698869 26721 raft_consensus.cc:2454] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a0b479ad79a43259b3a7c17ca09f4df in term 0.
I20260811 06:16:58.699664 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad9b94933d834f9e8c9078b84cd5ecf6" candidate_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a"
I20260811 06:16:58.699715 26498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad9b94933d834f9e8c9078b84cd5ecf6" candidate_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
I20260811 06:16:58.699857 26721 raft_consensus.cc:3037] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.699922 26498 raft_consensus.cc:3037] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:16:58.702894 26721 raft_consensus.cc:2454] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a0b479ad79a43259b3a7c17ca09f4df in term 1.
I20260811 06:16:58.699167 27123 leader_election.cc:288] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [CANDIDATE]: Term 1 election: Requested vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713)
I20260811 06:16:58.703684 26853 leader_election.cc:302] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a0b479ad79a43259b3a7c17ca09f4df, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:16:58.704156 26498 raft_consensus.cc:2454] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a0b479ad79a43259b3a7c17ca09f4df in term 1.
I20260811 06:16:58.704272 27123 raft_consensus.cc:2781] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:16:58.705057 27123 raft_consensus.cc:686] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 LEADER]: Becoming Leader. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Running, Role: LEADER
I20260811 06:16:58.705498 27123 consensus_queue.cc:237] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:16:58.709132 26219 catalog_manager.cc:5495] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3). New cstate: current_term: 1 leader_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260811 06:16:58.736394 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.728982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.22.121.3:35701)
0811 06:16:58.729137 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.729143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.729159 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:16:58.729243 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.729248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.729344 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.729484 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.730024 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.731127 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.733453 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.734085 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.734191 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.734742 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.735415 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.735537 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.736050 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.736129 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:16:58.755236 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.747908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:35025 (local address 127.22.121.1:43481)
0811 06:16:58.748075 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.748080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.749012 (+ 932us) server_negotiation.cc:407] Connection header received
0811 06:16:58.749221 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.749229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.749318 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.749387 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.750208 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.751566 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.753538 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.754022 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.754148 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.754583 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.754691 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.754765 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.754904 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.754980 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:16:58.755383 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.747486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.22.121.3:35025)
0811 06:16:58.748885 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.748905 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.749049 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.749597 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.749602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.749614 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:58.750016 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.750032 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.751809 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.751814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.753353 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.753369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.754205 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.754212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.754368 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.754417 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.754792 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.754803 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.754875 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.754996 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"thread_start_us":1079,"threads_started":1}
I20260811 06:16:58.763916 26498 raft_consensus.cc:1261] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Refusing update from remote peer 5a0b479ad79a43259b3a7c17ca09f4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.763984 26719 raft_consensus.cc:1261] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Refusing update from remote peer 5a0b479ad79a43259b3a7c17ca09f4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:16:58.764570 27123 consensus_queue.cc:1035] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.764846 27224 consensus_queue.cc:1035] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:16:58.770905 27166 mvcc.cc:205] Tried to move back new op lower bound from 7317213260851036160 to 7317213260621828096. Current Snapshot: MvccSnapshot[applied={T|T < 7317213260851036160}]
I20260811 06:16:58.793323 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.787426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:50855 (local address 127.22.121.3:35701)
0811 06:16:58.787588 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:16:58.787594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:16:58.787914 (+ 320us) server_negotiation.cc:407] Connection header received
0811 06:16:58.788069 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:16:58.788074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:16:58.788148 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:16:58.788240 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:16:58.788787 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.789631 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.791501 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.792183 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.792301 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.792723 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.792827 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.792893 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 06:16:58.793033 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 06:16:58.793102 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:16:58.793449 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:16:58.787360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35701 (local address 127.22.121.1:50855)
0811 06:16:58.787815 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 06:16:58.787832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:16:58.787953 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:16:58.788405 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:16:58.788411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:16:58.788424 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:16:58.788619 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.788630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.789753 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.789759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.791342 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:16:58.791358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:16:58.792323 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:16:58.792329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:16:58.792517 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:16:58.792568 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:16:58.792919 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:16:58.792931 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 06:16:58.793017 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 06:16:58.793185 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337}
W20260811 06:16:58.794430 27166 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:17:00.401036 26219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:00.401646 26219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:00.409054 26219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Sending DeleteTablet for 3 replicas of tablet ad9b94933d834f9e8c9078b84cd5ecf6
I20260811 06:17:00.410140 26460 tablet_service.cc:1514] Processing DeleteTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:00 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42242
I20260811 06:17:00.410490 26691 tablet_service.cc:1514] Processing DeleteTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:00 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:35768
I20260811 06:17:00.410624 26956 tablet_service.cc:1514] Processing DeleteTablet for tablet ad9b94933d834f9e8c9078b84cd5ecf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:00 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42760
I20260811 06:17:00.423578 27284 tablet_replica.cc:330] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: stopping tablet replica
I20260811 06:17:00.423658 27280 tablet_replica.cc:330] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: stopping tablet replica
I20260811 06:17:00.424150 27284 raft_consensus.cc:2229] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:00.424232 27280 raft_consensus.cc:2229] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:00.424657 27284 raft_consensus.cc:2258] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:00.424721 27280 raft_consensus.cc:2258] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:00.425765 27287 tablet_replica.cc:330] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: stopping tablet replica
I20260811 06:17:00.426164 27287 raft_consensus.cc:2229] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:00.426343 27284 ts_tablet_manager.cc:1905] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:00.426615 27280 ts_tablet_manager.cc:1905] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:00.427295 27287 raft_consensus.cc:2258] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:00.428912 27287 ts_tablet_manager.cc:1905] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:00.471040 27280 ts_tablet_manager.cc:1918] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:00.471129 27284 ts_tablet_manager.cc:1918] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:00.471484 27280 log.cc:1196] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/ad9b94933d834f9e8c9078b84cd5ecf6
I20260811 06:17:00.471473 27284 log.cc:1196] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/ad9b94933d834f9e8c9078b84cd5ecf6
I20260811 06:17:00.472258 27284 ts_tablet_manager.cc:1939] T ad9b94933d834f9e8c9078b84cd5ecf6 P 9a593961d4594e0da229b0efc538c79a: Deleting consensus metadata
I20260811 06:17:00.472683 27280 ts_tablet_manager.cc:1939] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting consensus metadata
Aug 11 06:17:00 dist-test-slave-gjrz krb5kdc[26136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260811 06:17:00.544824 26196 catalog_manager.cc:4841] TS 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481): tablet ad9b94933d834f9e8c9078b84cd5ecf6 (table test-table [id=4a5e054b7af14844915bb9c3bc8ab373]) successfully deleted
I20260811 06:17:00.544818 26189 catalog_manager.cc:4841] TS 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713): tablet ad9b94933d834f9e8c9078b84cd5ecf6 (table test-table [id=4a5e054b7af14844915bb9c3bc8ab373]) successfully deleted
W20260811 06:17:00.569554 23012 mini_kdc.cc:317] Time spent kinit for test-user: real 0.157s user 0.000s sys 0.036s
I20260811 06:17:00.569748 27287 ts_tablet_manager.cc:1918] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:00.570065 27287 log.cc:1196] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/ad9b94933d834f9e8c9078b84cd5ecf6
I20260811 06:17:00.570880 27287 ts_tablet_manager.cc:1939] T ad9b94933d834f9e8c9078b84cd5ecf6 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting consensus metadata
I20260811 06:17:00.573103 26196 catalog_manager.cc:4841] TS 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701): tablet ad9b94933d834f9e8c9078b84cd5ecf6 (table test-table [id=4a5e054b7af14844915bb9c3bc8ab373]) successfully deleted
Aug 11 06:17:00 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:00.598882 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:00.576042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.22.121.62:34159)
0811 06:17:00.577497 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:00.577508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:00.577537 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:17:00.577624 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:00.577628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:00.577714 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:00.577840 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:00.580485 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.581520 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.583290 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.584049 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.584216 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:00.590975 (+ 6759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:00.590982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:00.590985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:00.591029 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:00.595437 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:00.596418 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:00.596433 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:00.596436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:00.596651 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:00.597130 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:00.597133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:00.597135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:00.597585 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:00.597917 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:00.598218 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:17:00.598453 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":1117,"threads_started":1}
I20260811 06:17:00.604043 26219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:00.604784 26219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:00.614174 26460 tablet_service.cc:1467] Processing CreateTablet for tablet 0d60821a4e6644209220990d134d5c04 (DEFAULT_TABLE table=test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:00.614549 26691 tablet_service.cc:1467] Processing CreateTablet for tablet 0d60821a4e6644209220990d134d5c04 (DEFAULT_TABLE table=test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:00.615000 26460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d60821a4e6644209220990d134d5c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:00.614841 26956 tablet_service.cc:1467] Processing CreateTablet for tablet 0d60821a4e6644209220990d134d5c04 (DEFAULT_TABLE table=test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:00.615130 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d60821a4e6644209220990d134d5c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:00.615571 26956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d60821a4e6644209220990d134d5c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:00.623797 27302 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Bootstrap starting.
I20260811 06:17:00.626695 27302 tablet_bootstrap.cc:654] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:00.627339 27304 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Bootstrap starting.
I20260811 06:17:00.628479 27305 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Bootstrap starting.
I20260811 06:17:00.630748 27302 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: No bootstrap required, opened a new log
I20260811 06:17:00.630965 27302 ts_tablet_manager.cc:1397] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260811 06:17:00.631078 27304 tablet_bootstrap.cc:654] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:00.631417 27305 tablet_bootstrap.cc:654] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:00.632436 27302 raft_consensus.cc:348] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.632879 27302 raft_consensus.cc:374] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:00.633150 27302 raft_consensus.cc:729] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Initialized, Role: FOLLOWER
I20260811 06:17:00.633723 27302 consensus_queue.cc:260] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.635208 27305 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: No bootstrap required, opened a new log
I20260811 06:17:00.635464 27305 ts_tablet_manager.cc:1397] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260811 06:17:00.636276 27305 raft_consensus.cc:348] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.636540 27305 raft_consensus.cc:374] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:00.636649 27305 raft_consensus.cc:729] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Initialized, Role: FOLLOWER
I20260811 06:17:00.636870 27305 consensus_queue.cc:260] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.637553 27305 ts_tablet_manager.cc:1428] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 06:17:00.642232 27304 tablet_bootstrap.cc:492] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: No bootstrap required, opened a new log
I20260811 06:17:00.642300 27302 ts_tablet_manager.cc:1428] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260811 06:17:00.642529 27304 ts_tablet_manager.cc:1397] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260811 06:17:00.643710 27304 raft_consensus.cc:348] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.643988 27304 raft_consensus.cc:374] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:00.644140 27304 raft_consensus.cc:729] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Initialized, Role: FOLLOWER
I20260811 06:17:00.644515 27304 consensus_queue.cc:260] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.645238 27304 ts_tablet_manager.cc:1428] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:00.808153 27116 raft_consensus.cc:482] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:00.808481 27116 raft_consensus.cc:504] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.809674 27116 leader_election.cc:288] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:17:00.810544 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d60821a4e6644209220990d134d5c04" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a" is_pre_election: true
I20260811 06:17:00.810572 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d60821a4e6644209220990d134d5c04" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" is_pre_election: true
I20260811 06:17:00.810918 26984 raft_consensus.cc:2454] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 0.
I20260811 06:17:00.810946 26721 raft_consensus.cc:2454] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 0.
I20260811 06:17:00.811728 26381 leader_election.cc:302] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 5a0b479ad79a43259b3a7c17ca09f4df; no voters:
I20260811 06:17:00.812238 27116 raft_consensus.cc:2781] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:00.812448 27116 raft_consensus.cc:482] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:00.812606 27116 raft_consensus.cc:3037] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:00.816711 27116 raft_consensus.cc:504] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.817651 27116 leader_election.cc:288] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 election: Requested vote from peers 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:17:00.818501 26721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d60821a4e6644209220990d134d5c04" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a593961d4594e0da229b0efc538c79a"
I20260811 06:17:00.818591 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d60821a4e6644209220990d134d5c04" candidate_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
I20260811 06:17:00.819007 26984 raft_consensus.cc:3037] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:00.819139 26721 raft_consensus.cc:3037] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:00.825234 26984 raft_consensus.cc:2454] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 1.
I20260811 06:17:00.825271 26721 raft_consensus.cc:2454] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5537732d10ba4a248b14e6ad5b8f5b25 in term 1.
I20260811 06:17:00.825973 26374 leader_election.cc:302] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 5a0b479ad79a43259b3a7c17ca09f4df, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:17:00.826474 27116 raft_consensus.cc:2781] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:00.826736 27116 raft_consensus.cc:686] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 LEADER]: Becoming Leader. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Running, Role: LEADER
I20260811 06:17:00.827167 27116 consensus_queue.cc:237] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:00.832006 26222 catalog_manager.cc:5495] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260811 06:17:00.888252 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:00.879196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.22.121.1:43481)
0811 06:17:00.880206 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:00.880211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:00.880234 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:17:00.880310 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:00.880314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:00.880428 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:00.880583 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:00.881363 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.882813 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.885200 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.885975 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.886208 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:00.886931 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:00.887135 (+ 204us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:00.887299 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:00.887552 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:17:00.887700 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":534,"threads_started":1}
I20260811 06:17:00.910662 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:00.901613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43821 (local address 127.22.121.1:43481)
0811 06:17:00.901917 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:00.901924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:00.902738 (+ 814us) server_negotiation.cc:407] Connection header received
0811 06:17:00.903060 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:00.903066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:00.903185 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:00.903335 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:00.904402 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.905640 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.907878 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.908546 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.908752 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:00.909436 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:00.909593 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:00.909749 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:00.909936 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 06:17:00.910100 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":74}
I20260811 06:17:00.910857 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:00.901359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43481 (local address 127.22.121.1:43821)
0811 06:17:00.902601 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:00.902629 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:00.902892 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:00.903348 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:00.903356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:00.903378 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:00.904095 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:00.904125 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.905858 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.905867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:00.907697 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:00.907720 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:00.908804 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:00.908821 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:00.909135 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:00.909198 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:00.909745 (+ 547us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:00.909769 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:17:00.909917 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 06:17:00.910158 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":648,"threads_started":1}
I20260811 06:17:00.923913 26721 raft_consensus.cc:1261] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Refusing update from remote peer 5537732d10ba4a248b14e6ad5b8f5b25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:00.924304 26984 raft_consensus.cc:1261] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Refusing update from remote peer 5537732d10ba4a248b14e6ad5b8f5b25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:00.924986 27327 consensus_queue.cc:1035] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 06:17:00.925663 27246 consensus_queue.cc:1035] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:01.018591 26222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:01.019121 26222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:01.025589 26222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Sending DeleteTablet for 3 replicas of tablet 0d60821a4e6644209220990d134d5c04
I20260811 06:17:01.027238 26691 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d60821a4e6644209220990d134d5c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:01 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:35768
I20260811 06:17:01.027484 26956 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d60821a4e6644209220990d134d5c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:01 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42760
I20260811 06:17:01.027412 26460 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d60821a4e6644209220990d134d5c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:01 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42242
I20260811 06:17:01.027891 27284 tablet_replica.cc:330] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: stopping tablet replica
I20260811 06:17:01.028072 27287 tablet_replica.cc:330] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: stopping tablet replica
I20260811 06:17:01.028079 27280 tablet_replica.cc:330] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: stopping tablet replica
I20260811 06:17:01.028352 27284 raft_consensus.cc:2229] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:01.028558 27287 raft_consensus.cc:2229] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:01.028585 27280 raft_consensus.cc:2229] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:01.028755 27284 raft_consensus.cc:2258] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:01.028975 27287 raft_consensus.cc:2258] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:01.029673 27280 raft_consensus.cc:2258] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:01.031620 27284 ts_tablet_manager.cc:1905] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:01.031849 27287 ts_tablet_manager.cc:1905] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:01.031975 27280 ts_tablet_manager.cc:1905] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:01.038265 26219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:01.039614 26219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:01.045156 27287 ts_tablet_manager.cc:1918] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:01.045670 27287 log.cc:1196] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/0d60821a4e6644209220990d134d5c04
I20260811 06:17:01.047150 27287 ts_tablet_manager.cc:1939] T 0d60821a4e6644209220990d134d5c04 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting consensus metadata
I20260811 06:17:01.049381 26196 catalog_manager.cc:4841] TS 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701): tablet 0d60821a4e6644209220990d134d5c04 (table test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]) successfully deleted
I20260811 06:17:01.056504 26956 tablet_service.cc:1467] Processing CreateTablet for tablet 00838512403b4f1fb020136876a864c0 (DEFAULT_TABLE table=test-table [id=34fd72c388f44d86ad759648789be43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:01.057410 26956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00838512403b4f1fb020136876a864c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:01.058758 27280 ts_tablet_manager.cc:1918] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:01.059175 27280 log.cc:1196] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/0d60821a4e6644209220990d134d5c04
I20260811 06:17:01.059646 26691 tablet_service.cc:1467] Processing CreateTablet for tablet 00838512403b4f1fb020136876a864c0 (DEFAULT_TABLE table=test-table [id=34fd72c388f44d86ad759648789be43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:01.060672 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00838512403b4f1fb020136876a864c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:01.061981 26460 tablet_service.cc:1467] Processing CreateTablet for tablet 00838512403b4f1fb020136876a864c0 (DEFAULT_TABLE table=test-table [id=34fd72c388f44d86ad759648789be43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:01.062781 26460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00838512403b4f1fb020136876a864c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:01.064535 27280 ts_tablet_manager.cc:1939] T 0d60821a4e6644209220990d134d5c04 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting consensus metadata
I20260811 06:17:01.068497 26196 catalog_manager.cc:4841] TS 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481): tablet 0d60821a4e6644209220990d134d5c04 (table test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]) successfully deleted
I20260811 06:17:01.072793 27305 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Bootstrap starting.
I20260811 06:17:01.073858 27284 ts_tablet_manager.cc:1918] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:01.077126 27304 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Bootstrap starting.
I20260811 06:17:01.077277 27305 tablet_bootstrap.cc:654] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:01.077620 27284 log.cc:1196] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/0d60821a4e6644209220990d134d5c04
I20260811 06:17:01.078177 27284 ts_tablet_manager.cc:1939] T 0d60821a4e6644209220990d134d5c04 P 9a593961d4594e0da229b0efc538c79a: Deleting consensus metadata
I20260811 06:17:01.081269 26189 catalog_manager.cc:4841] TS 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713): tablet 0d60821a4e6644209220990d134d5c04 (table test-table [id=ec7076ab89d6420f8877afb97cc2a8fb]) successfully deleted
I20260811 06:17:01.082854 27305 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: No bootstrap required, opened a new log
I20260811 06:17:01.083438 27305 ts_tablet_manager.cc:1397] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260811 06:17:01.083702 27304 tablet_bootstrap.cc:654] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:01.085858 27305 raft_consensus.cc:348] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.086537 27305 raft_consensus.cc:374] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:01.086859 27305 raft_consensus.cc:729] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0b479ad79a43259b3a7c17ca09f4df, State: Initialized, Role: FOLLOWER
I20260811 06:17:01.087538 27305 consensus_queue.cc:260] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.090201 27305 ts_tablet_manager.cc:1428] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260811 06:17:01.090978 27302 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Bootstrap starting.
I20260811 06:17:01.093881 27304 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: No bootstrap required, opened a new log
I20260811 06:17:01.094365 27304 ts_tablet_manager.cc:1397] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.005s
I20260811 06:17:01.097223 27302 tablet_bootstrap.cc:654] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:01.097347 27304 raft_consensus.cc:348] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.097973 27304 raft_consensus.cc:374] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:01.098212 27304 raft_consensus.cc:729] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5537732d10ba4a248b14e6ad5b8f5b25, State: Initialized, Role: FOLLOWER
I20260811 06:17:01.098629 27304 consensus_queue.cc:260] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.099989 27304 ts_tablet_manager.cc:1428] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 06:17:01.102996 27302 tablet_bootstrap.cc:492] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: No bootstrap required, opened a new log
I20260811 06:17:01.103400 27302 ts_tablet_manager.cc:1397] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260811 06:17:01.105201 27302 raft_consensus.cc:348] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.105507 27302 raft_consensus.cc:374] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:01.105659 27302 raft_consensus.cc:729] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Initialized, Role: FOLLOWER
I20260811 06:17:01.106081 27302 consensus_queue.cc:260] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.107344 27302 ts_tablet_manager.cc:1428] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 06:17:01.268872 27312 raft_consensus.cc:482] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:01.269348 27312 raft_consensus.cc:504] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.270717 27312 leader_election.cc:288] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:17:01.271786 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00838512403b4f1fb020136876a864c0" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" is_pre_election: true
I20260811 06:17:01.272369 26496 raft_consensus.cc:2454] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 0.
I20260811 06:17:01.272255 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00838512403b4f1fb020136876a864c0" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" is_pre_election: true
I20260811 06:17:01.272713 26984 raft_consensus.cc:2454] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 0.
I20260811 06:17:01.273245 26607 leader_election.cc:302] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:17:01.273703 27312 raft_consensus.cc:2781] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:01.273902 27312 raft_consensus.cc:482] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:01.274102 27312 raft_consensus.cc:3037] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:01.277613 27312 raft_consensus.cc:504] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.278628 27312 leader_election.cc:288] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 election: Requested vote from peers 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481), 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701)
I20260811 06:17:01.279358 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00838512403b4f1fb020136876a864c0" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5537732d10ba4a248b14e6ad5b8f5b25"
I20260811 06:17:01.279650 26496 raft_consensus.cc:3037] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:01.279503 26984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00838512403b4f1fb020136876a864c0" candidate_uuid: "9a593961d4594e0da229b0efc538c79a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0b479ad79a43259b3a7c17ca09f4df"
I20260811 06:17:01.279861 26984 raft_consensus.cc:3037] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:01.283200 26496 raft_consensus.cc:2454] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 1.
I20260811 06:17:01.283891 26607 leader_election.cc:302] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5537732d10ba4a248b14e6ad5b8f5b25, 9a593961d4594e0da229b0efc538c79a; no voters:
I20260811 06:17:01.284443 27312 raft_consensus.cc:2781] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:01.285399 27312 raft_consensus.cc:686] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 1 LEADER]: Becoming Leader. State: Replica: 9a593961d4594e0da229b0efc538c79a, State: Running, Role: LEADER
I20260811 06:17:01.285689 26984 raft_consensus.cc:2454] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a593961d4594e0da229b0efc538c79a in term 1.
I20260811 06:17:01.286198 27312 consensus_queue.cc:237] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } }
I20260811 06:17:01.291666 26219 catalog_manager.cc:5495] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a593961d4594e0da229b0efc538c79a (127.22.121.2). New cstate: current_term: 1 leader_uuid: "9a593961d4594e0da229b0efc538c79a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a593961d4594e0da229b0efc538c79a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 41713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260811 06:17:01.348155 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:01.337089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46628 (local address 127.22.121.2:41713)
0811 06:17:01.338297 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:01.338303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:01.338339 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:17:01.338419 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:01.338423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:01.338552 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:01.338689 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:01.339669 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:01.341392 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:01.344922 (+ 3530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:01.345666 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:01.345827 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:01.346848 (+ 1021us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:01.347034 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:01.347183 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:01.347500 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 06:17:01.347614 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":809,"threads_started":1}
I20260811 06:17:01.364843 26984 raft_consensus.cc:1261] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Refusing update from remote peer 9a593961d4594e0da229b0efc538c79a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:01.364914 26496 raft_consensus.cc:1261] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Refusing update from remote peer 9a593961d4594e0da229b0efc538c79a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:01.365809 27312 consensus_queue.cc:1035] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5537732d10ba4a248b14e6ad5b8f5b25" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:01.366346 27344 consensus_queue.cc:1035] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a0b479ad79a43259b3a7c17ca09f4df" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:17:01.406400 27249 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:17:03.046938 26222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:03.047431 26222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:03.054644 26222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd17551e4d34b1a95104d75488c35db: Sending DeleteTablet for 3 replicas of tablet 00838512403b4f1fb020136876a864c0
I20260811 06:17:03.055856 26691 tablet_service.cc:1514] Processing DeleteTablet for tablet 00838512403b4f1fb020136876a864c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:03 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:35768
I20260811 06:17:03.055927 26460 tablet_service.cc:1514] Processing DeleteTablet for tablet 00838512403b4f1fb020136876a864c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:03 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42242
I20260811 06:17:03.055927 26956 tablet_service.cc:1514] Processing DeleteTablet for tablet 00838512403b4f1fb020136876a864c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:03 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42760
I20260811 06:17:03.060629 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26348
I20260811 06:17:03.062232 27414 tablet_replica.cc:330] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: stopping tablet replica
I20260811 06:17:03.063239 27410 tablet_replica.cc:330] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: stopping tablet replica
I20260811 06:17:03.063406 27413 tablet_replica.cc:330] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: stopping tablet replica
I20260811 06:17:03.063479 27414 raft_consensus.cc:2229] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:03.063862 27413 raft_consensus.cc:2229] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:03.063752 27410 raft_consensus.cc:2229] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:03.064224 27414 raft_consensus.cc:2258] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:03.064453 27410 raft_consensus.cc:2258] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:03.064577 27413 raft_consensus.cc:2258] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:03.066046 27414 ts_tablet_manager.cc:1905] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:03.066087 27410 ts_tablet_manager.cc:1905] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:03.066345 27413 ts_tablet_manager.cc:1905] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:03.077332 27414 ts_tablet_manager.cc:1918] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:03.077642 27414 log.cc:1196] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/00838512403b4f1fb020136876a864c0
I20260811 06:17:03.077762 27413 ts_tablet_manager.cc:1918] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:03.077981 27410 ts_tablet_manager.cc:1918] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:17:03.078037 27413 log.cc:1196] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/00838512403b4f1fb020136876a864c0
I20260811 06:17:03.078315 27410 log.cc:1196] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/00838512403b4f1fb020136876a864c0
I20260811 06:17:03.078583 27414 ts_tablet_manager.cc:1939] T 00838512403b4f1fb020136876a864c0 P 5a0b479ad79a43259b3a7c17ca09f4df: Deleting consensus metadata
I20260811 06:17:03.078945 27413 ts_tablet_manager.cc:1939] T 00838512403b4f1fb020136876a864c0 P 9a593961d4594e0da229b0efc538c79a: Deleting consensus metadata
I20260811 06:17:03.079372 27410 ts_tablet_manager.cc:1939] T 00838512403b4f1fb020136876a864c0 P 5537732d10ba4a248b14e6ad5b8f5b25: Deleting consensus metadata
I20260811 06:17:03.080577 26196 catalog_manager.cc:4841] TS 5a0b479ad79a43259b3a7c17ca09f4df (127.22.121.3:35701): tablet 00838512403b4f1fb020136876a864c0 (table test-table [id=34fd72c388f44d86ad759648789be43e]) successfully deleted
I20260811 06:17:03.080821 26189 catalog_manager.cc:4841] TS 9a593961d4594e0da229b0efc538c79a (127.22.121.2:41713): tablet 00838512403b4f1fb020136876a864c0 (table test-table [id=34fd72c388f44d86ad759648789be43e]) successfully deleted
I20260811 06:17:03.081110 26196 catalog_manager.cc:4841] TS 5537732d10ba4a248b14e6ad5b8f5b25 (127.22.121.1:43481): tablet 00838512403b4f1fb020136876a864c0 (table test-table [id=34fd72c388f44d86ad759648789be43e]) successfully deleted
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:03.103291 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:03.062808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59106 (local address 127.22.121.1:43481)
0811 06:17:03.081725 (+ 18917us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:03.081729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:03.081918 (+ 189us) server_negotiation.cc:407] Connection header received
0811 06:17:03.082150 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:03.082155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:03.082245 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:03.082379 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:03.084435 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.085669 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.087702 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.088376 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.088536 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:03.094650 (+ 6114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:03.094675 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:03.094704 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:03.094755 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:03.099191 (+ 4436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.100434 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.100451 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.100454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.100693 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.101232 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.101235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.101237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.101903 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:03.102248 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:03.102562 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:17:03.102838 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18754,"thread_start_us":18553,"threads_started":1}
W20260811 06:17:03.104278 26562 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59106
W20260811 06:17:03.105214 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26348: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:03.105495 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26348
I20260811 06:17:03.184326 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26575
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:03.215623 27418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:03.185505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.22.121.2:41713)
0811 06:17:03.186338 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:03.186344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:03.186370 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:17:03.186460 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:03.186469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:03.186577 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:03.186757 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:03.188997 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.190276 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.192692 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.193703 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.194009 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:03.203805 (+ 9796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:03.203830 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:03.203858 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:03.203913 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:03.209974 (+ 6061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.211710 (+ 1736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.211732 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.211746 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.212094 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.213020 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.213032 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.213034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.213701 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:03.214149 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:03.214707 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 06:17:03.214994 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":392,"threads_started":1}
W20260811 06:17:03.216712 26804 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46636
W20260811 06:17:03.217700 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26575: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:03.217976 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26575
I20260811 06:17:03.271024 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26826
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[26136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:03.300377 27420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:03.272300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.22.121.3:35701)
0811 06:17:03.273327 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:03.273334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:03.273370 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:17:03.273461 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:03.273474 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:03.273583 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:03.273738 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:03.276770 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.278309 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.280848 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:03.281794 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:03.282018 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:03.290599 (+ 8581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:03.290634 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:03.290670 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:03.290748 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:03.296204 (+ 5456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.297388 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.297408 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.297410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.297691 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:03.298159 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:03.298162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:03.298164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:03.298765 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:03.299152 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:03.299548 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:17:03.299797 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":531,"threads_started":1}
W20260811 06:17:03.301357 27070 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56042
W20260811 06:17:03.302300 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26826: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:03.302505 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26826
I20260811 06:17:03.368537 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26158
I20260811 06:17:03.370180 26292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:03.519593 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26158
2026-08-11T06:17:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9146 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[27426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[27426](info): set up 2 sockets
Aug 11 06:17:03 dist-test-slave-gjrz krb5kdc[27426](info): commencing operation
krb5kdc: starting...
W20260811 06:17:05.228197 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.639s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:05 dist-test-slave-gjrz krb5kdc[27426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:05.398002 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39521
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:05.702739 27447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:05.702988 27447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:05.703120 27447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:05.709957 27447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:05.710059 27447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:05.710136 27447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:05.710209 27447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:05.710268 27447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:05.719364 27447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:05.720455 27447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:05.721611 27447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:05.732204 27458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:05.732337 27457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:05.732250 27462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:05.733142 27447 server_base.cc:1034] running on GCE node
I20260811 06:17:05.733789 27447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:05.734992 27447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:05.736191 27447 hybrid_clock.cc:648] HybridClock initialized: now 1786429025736146 us; error 25 us; skew 500 ppm
I20260811 06:17:05.736501 27447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:05 dist-test-slave-gjrz krb5kdc[27426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:05.739316 27447 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:05.740710 27447 webserver.cc:460] Webserver started at http://127.22.121.62:39747/ using document root <none> and password file <none>
I20260811 06:17:05.741320 27447 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:05.741425 27447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:05.741700 27447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:05.744438 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "418946dd5e484b36b3541cabbbdf35c6"
format_stamp: "Formatted at 2026-08-11 06:17:05 on dist-test-slave-gjrz"
server_key: "c91fad6a1e3ec5695785328f93552723"
server_key_iv: "e07b34f9253260ecdf46bff7025ed02a"
server_key_version: "encryptionkey@0"
I20260811 06:17:05.745151 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "418946dd5e484b36b3541cabbbdf35c6"
format_stamp: "Formatted at 2026-08-11 06:17:05 on dist-test-slave-gjrz"
server_key: "c91fad6a1e3ec5695785328f93552723"
server_key_iv: "e07b34f9253260ecdf46bff7025ed02a"
server_key_version: "encryptionkey@0"
I20260811 06:17:05.749836 27447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:17:05.752961 27479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:05.754079 27447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:05.754231 27447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "418946dd5e484b36b3541cabbbdf35c6"
format_stamp: "Formatted at 2026-08-11 06:17:05 on dist-test-slave-gjrz"
server_key: "c91fad6a1e3ec5695785328f93552723"
server_key_iv: "e07b34f9253260ecdf46bff7025ed02a"
server_key_version: "encryptionkey@0"
I20260811 06:17:05.754397 27447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:05.777714 27447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:05.781189 27447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:05.781430 27447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:05.797458 27447 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:39521
I20260811 06:17:05.797479 27567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:39521 every 8 connection(s)
I20260811 06:17:05.798612 27447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:05.801173 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27447
I20260811 06:17:05.801374 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:05.802217 23012 external_mini_cluster.cc:1428] Setting key e33587403414ef437daf18a5b97f0d09
I20260811 06:17:05.802383 27569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:05.815742 27569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6: Bootstrap starting.
Aug 11 06:17:05 dist-test-slave-gjrz krb5kdc[27426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:05.818703 27569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:05.819725 27569 log.cc:824] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:05.822141 27569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6: No bootstrap required, opened a new log
I20260811 06:17:05.823508 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:05.804075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.22.121.62:39521)
0811 06:17:05.804762 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:05.804773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:05.804813 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:17:05.806720 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:05.806740 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:05.807099 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:05.807440 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:05.809058 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:05.810293 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:05.811999 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:05.812679 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:05.812800 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:05.817081 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:05.817089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:05.817103 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:05.817141 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:05.819949 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:05.820588 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:05.820601 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:05.820607 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:05.820761 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:05.821110 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:05.821114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:05.821116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:05.821556 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:05.821818 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:05.822245 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 06:17:05.822608 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":308,"threads_started":1}
I20260811 06:17:05.829615 27569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } }
I20260811 06:17:05.829917 27569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:05.830004 27569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418946dd5e484b36b3541cabbbdf35c6, State: Initialized, Role: FOLLOWER
I20260811 06:17:05.830503 27569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } }
I20260811 06:17:05.830726 27569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:05.830847 27569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:05.830989 27569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:05.832994 27569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } }
I20260811 06:17:05.833475 27569 leader_election.cc:302] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 418946dd5e484b36b3541cabbbdf35c6; no voters:
I20260811 06:17:05.834089 27569 leader_election.cc:288] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:05.834203 27582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:05.834774 27582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [term 1 LEADER]: Becoming Leader. State: Replica: 418946dd5e484b36b3541cabbbdf35c6, State: Running, Role: LEADER
I20260811 06:17:05.835315 27582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } }
I20260811 06:17:05.836155 27569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:05.838057 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 418946dd5e484b36b3541cabbbdf35c6. Latest consensus state: current_term: 1 leader_uuid: "418946dd5e484b36b3541cabbbdf35c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } } }
I20260811 06:17:05.838091 27583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "418946dd5e484b36b3541cabbbdf35c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418946dd5e484b36b3541cabbbdf35c6" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39521 } } }
I20260811 06:17:05.838424 27583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:05.838516 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:05.842161 27592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:05.846920 27592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:05.855109 27592 catalog_manager.cc:1347] Generated new cluster ID: 0c1fcb9d14f44d5390f74b1ab20bd3ed
I20260811 06:17:05.855199 27592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:05.870121 27592 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:05.871245 27592 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:05.876698 27592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 418946dd5e484b36b3541cabbbdf35c6: Generated new TSK 0
I20260811 06:17:05.877305 27592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:05.926239 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:06.265969 27629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:06.266212 27629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:06.266338 27629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:06.273183 27629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:06.273274 27629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:06.273412 27629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:06.282424 27629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:06.283528 27629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:06.284701 27629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:06.296765 27639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:06.296803 27640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:06.296836 27645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:06.297473 27629 server_base.cc:1034] running on GCE node
I20260811 06:17:06.298048 27629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:06.298869 27629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:06.300060 27629 hybrid_clock.cc:648] HybridClock initialized: now 1786429026300006 us; error 45 us; skew 500 ppm
I20260811 06:17:06.300347 27629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:06 dist-test-slave-gjrz krb5kdc[27426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429026, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:06.303231 27629 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:06.304555 27629 webserver.cc:460] Webserver started at http://127.22.121.1:34533/ using document root <none> and password file <none>
I20260811 06:17:06.305150 27629 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:06.305245 27629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:06.305492 27629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:06.308166 27629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "cad132722b184736b34921b2e760efec"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "f0a6360f7bcf55fe3e71985be8a0da60"
server_key_iv: "a3b041c27045bac5a11eb58e127ec482"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.308877 27629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "cad132722b184736b34921b2e760efec"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "f0a6360f7bcf55fe3e71985be8a0da60"
server_key_iv: "a3b041c27045bac5a11eb58e127ec482"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.313499 27629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 06:17:06.316597 27661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.317742 27629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:06.317860 27629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "cad132722b184736b34921b2e760efec"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "f0a6360f7bcf55fe3e71985be8a0da60"
server_key_iv: "a3b041c27045bac5a11eb58e127ec482"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.318004 27629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:06.337158 27629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:06.340979 27629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:06.341197 27629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:06.342010 27629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:06.343175 27629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:06.343250 27629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.343389 27629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:06.343437 27629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.372277 27629 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:35807
I20260811 06:17:06.372335 27861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:35807 every 8 connection(s)
I20260811 06:17:06.373412 27629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:06.374030 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27629
I20260811 06:17:06.374214 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:06.375048 23012 external_mini_cluster.cc:1428] Setting key da8c1c2551e57fd4145bb271c28af04a
Aug 11 06:17:06 dist-test-slave-gjrz krb5kdc[27426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429026, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:06.392532 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:06.376254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:38553 (local address 127.22.121.62:39521)
0811 06:17:06.377266 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:06.377272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:06.377850 (+ 578us) server_negotiation.cc:407] Connection header received
0811 06:17:06.379616 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:06.379621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:06.379708 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:06.379800 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:06.381166 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.381872 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.383371 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.383783 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.383890 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:06.387233 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:06.387238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:06.387241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:06.387296 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:06.389663 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:06.390373 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:06.390377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:06.390379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:06.390495 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:06.390878 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:06.390883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:06.390886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:06.391169 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:06.391344 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:06.392139 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 06:17:06.392294 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":779,"threads_started":1}
I20260811 06:17:06.393839 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:06.376674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39521 (local address 127.22.121.1:38553)
0811 06:17:06.377711 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:06.377752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:06.379381 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:06.380038 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:06.380046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:06.380503 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:06.381011 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:06.381029 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.382042 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.382046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:06.383225 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:06.383237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.384002 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.384010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:06.384221 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:06.384764 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:06.384791 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:06.387038 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:06.389790 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:06.389799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:06.389813 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:06.390263 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:06.390596 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:06.390601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:06.390605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:06.390775 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:06.391359 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:06.391367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:06.391770 (+ 403us) client_negotiation.cc:769] Sending connection context
0811 06:17:06.392082 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 06:17:06.392570 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":602,"threads_started":1}
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:17:06.395190 27862 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39521
I20260811 06:17:06.395467 27862 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:06.396164 27862 heartbeater.cc:511] Master 127.22.121.62:39521 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
I20260811 06:17:06.398129 27500 ts_manager.cc:194] Registered new tserver with Master: cad132722b184736b34921b2e760efec (127.22.121.1:35807)
I20260811 06:17:06.399569 27500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:38553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:06.416394 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:06.720431 27872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:06.720683 27872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:06.720795 27872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:06.727650 27872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:06.727735 27872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:06.727842 27872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:06.736788 27872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:06.737874 27872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:06.739010 27872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:06.750913 27881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:06.750952 27886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:06.750979 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:06.751796 27872 server_base.cc:1034] running on GCE node
I20260811 06:17:06.752290 27872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:06.753068 27872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:06.754257 27872 hybrid_clock.cc:648] HybridClock initialized: now 1786429026754207 us; error 45 us; skew 500 ppm
I20260811 06:17:06.754506 27872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:06 dist-test-slave-gjrz krb5kdc[27426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429026, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:06.757334 27872 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:06.758603 27872 webserver.cc:460] Webserver started at http://127.22.121.2:38229/ using document root <none> and password file <none>
I20260811 06:17:06.759217 27872 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:06.759346 27872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:06.759666 27872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:06.762264 27872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "dc7af6b3321541ccbbd969dea6808339"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "c0a4a459366569d72a89baac7038e3db"
server_key_iv: "ddf42eb7452b2782f1ecd42e2d06648d"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.762971 27872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "dc7af6b3321541ccbbd969dea6808339"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "c0a4a459366569d72a89baac7038e3db"
server_key_iv: "ddf42eb7452b2782f1ecd42e2d06648d"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.767607 27872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:17:06.770607 27899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.771767 27872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:06.771909 27872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "dc7af6b3321541ccbbd969dea6808339"
format_stamp: "Formatted at 2026-08-11 06:17:06 on dist-test-slave-gjrz"
server_key: "c0a4a459366569d72a89baac7038e3db"
server_key_iv: "ddf42eb7452b2782f1ecd42e2d06648d"
server_key_version: "encryptionkey@0"
I20260811 06:17:06.772071 27872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:06.800437 27872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:06.804960 27872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:06.805227 27872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:06.806213 27872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:06.807654 27872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:06.807736 27872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.807835 27872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:06.807888 27872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:06.837563 27872 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:40101
I20260811 06:17:06.837622 28118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:40101 every 8 connection(s)
I20260811 06:17:06.838657 27872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:06.840306 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27872
I20260811 06:17:06.840471 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:06.840969 23012 external_mini_cluster.cc:1428] Setting key ea8e8e731c4f43fd00a390865a12c9f1
Aug 11 06:17:06 dist-test-slave-gjrz krb5kdc[27426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429026, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:06.858158 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:06.841118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:39551 (local address 127.22.121.62:39521)
0811 06:17:06.841310 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:06.841317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:06.842494 (+ 1177us) server_negotiation.cc:407] Connection header received
0811 06:17:06.844301 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:06.844307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:06.844384 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:06.844482 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:06.845864 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.846952 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.848418 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.848982 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.849103 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:06.852651 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:06.852655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:06.852658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:06.852688 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:06.855060 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:06.855844 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:06.855848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:06.855850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:06.855973 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:06.856441 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:06.856445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:06.856447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:06.856794 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:06.856920 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:06.857697 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 06:17:06.857880 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:06.859458 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:06.841525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39521 (local address 127.22.121.2:39551)
0811 06:17:06.842332 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:06.842383 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:06.844060 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:06.844674 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:06.844681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:06.845175 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:06.845698 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:06.845723 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.847075 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.847078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:06.848288 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:06.848300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:06.849154 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:06.849158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:06.849340 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:06.849954 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:06.849981 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:06.852445 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:06.855188 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:06.855196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:06.855209 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:06.855740 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:06.856069 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:06.856073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:06.856077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:06.856290 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:06.856924 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:06.856931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:06.857314 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 06:17:06.857627 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 06:17:06.858121 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":346,"threads_started":1}
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
I20260811 06:17:06.860782 28121 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39521
I20260811 06:17:06.861042 28121 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:06.861693 28121 heartbeater.cc:511] Master 127.22.121.62:39521 requested a full tablet report, sending...
I20260811 06:17:06.863225 27500 ts_manager.cc:194] Registered new tserver with Master: dc7af6b3321541ccbbd969dea6808339 (127.22.121.2:40101)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
I20260811 06:17:06.864111 27500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:39551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:06.883663 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:07.187856 28131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:07.188102 28131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:07.188230 28131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:07.195171 28131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:07.195282 28131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:07.195432 28131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:07.204491 28131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:07.205556 28131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:07.206763 28131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:07.218636 28143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:07.218674 28138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:07.218709 28141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:07.219578 28131 server_base.cc:1034] running on GCE node
I20260811 06:17:07.220153 28131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:07.221019 28131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:07.222214 28131 hybrid_clock.cc:648] HybridClock initialized: now 1786429027222169 us; error 43 us; skew 500 ppm
I20260811 06:17:07.222493 28131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:07 dist-test-slave-gjrz krb5kdc[27426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429027, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:07.225395 28131 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:07.226694 28131 webserver.cc:460] Webserver started at http://127.22.121.3:38537/ using document root <none> and password file <none>
I20260811 06:17:07.227335 28131 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:07.227455 28131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:07.227748 28131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:07.230501 28131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "b926b2c269bd4dc9bdedecfaf6d4f6ef"
format_stamp: "Formatted at 2026-08-11 06:17:07 on dist-test-slave-gjrz"
server_key: "7e1dcd4ef42c9038dfd0e1e1a14c6ab2"
server_key_iv: "b5778f2019b464eeaffb5e90f838d7f9"
server_key_version: "encryptionkey@0"
I20260811 06:17:07.231253 28131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "b926b2c269bd4dc9bdedecfaf6d4f6ef"
format_stamp: "Formatted at 2026-08-11 06:17:07 on dist-test-slave-gjrz"
server_key: "7e1dcd4ef42c9038dfd0e1e1a14c6ab2"
server_key_iv: "b5778f2019b464eeaffb5e90f838d7f9"
server_key_version: "encryptionkey@0"
I20260811 06:17:07.236234 28131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:17:07.239601 28151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:07.240772 28131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:07.240912 28131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "b926b2c269bd4dc9bdedecfaf6d4f6ef"
format_stamp: "Formatted at 2026-08-11 06:17:07 on dist-test-slave-gjrz"
server_key: "7e1dcd4ef42c9038dfd0e1e1a14c6ab2"
server_key_iv: "b5778f2019b464eeaffb5e90f838d7f9"
server_key_version: "encryptionkey@0"
I20260811 06:17:07.241088 28131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:07.261124 28131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:07.264559 28131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:07.264818 28131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:07.265677 28131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:07.266887 28131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:07.266969 28131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:07.267082 28131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:07.267136 28131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:07.296182 28131 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:38463
I20260811 06:17:07.296241 28371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:38463 every 8 connection(s)
I20260811 06:17:07.297329 28131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:07.307360 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28131
I20260811 06:17:07.307538 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:07.308090 23012 external_mini_cluster.cc:1428] Setting key 5437e764de06ba12f5facbcb8b664098
Aug 11 06:17:07 dist-test-slave-gjrz krb5kdc[27426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429027, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:07.315522 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:07.299615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:43777 (local address 127.22.121.62:39521)
0811 06:17:07.299811 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:07.299817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:07.300897 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 06:17:07.302612 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:07.302617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:07.302682 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:07.302763 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:07.304152 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:07.304930 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:07.306369 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:07.306769 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:07.306857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:07.310622 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:07.310627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:07.310631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:07.310669 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:07.313201 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:07.313805 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:07.313808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:07.313811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:07.313917 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:07.314230 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:07.314233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:07.314235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:07.314473 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:07.314575 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:07.315104 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 06:17:07.315249 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:17:07.316381 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:07.300060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39521 (local address 127.22.121.3:43777)
0811 06:17:07.300741 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:07.300791 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:07.302381 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:07.302948 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:07.302955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:07.303455 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:07.303994 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:07.304010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:07.305051 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:07.305054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:07.306257 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:07.306270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:07.306937 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:07.306944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:07.307223 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:07.307896 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:07.307932 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:07.310427 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:07.313310 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:07.313316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:07.313326 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:07.313720 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:07.313988 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:07.313992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:07.313994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:07.314155 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:07.314558 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:07.314563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:07.314839 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 06:17:07.315047 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 06:17:07.315438 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":247,"threads_started":1}
I20260811 06:17:07.317554 28372 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39521
I20260811 06:17:07.317816 28372 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:07.318437 28372 heartbeater.cc:511] Master 127.22.121.62:39521 requested a full tablet report, sending...
I20260811 06:17:07.319717 27500 ts_manager.cc:194] Registered new tserver with Master: b926b2c269bd4dc9bdedecfaf6d4f6ef (127.22.121.3:38463)
I20260811 06:17:07.320441 27500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:43777
I20260811 06:17:07.322989 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:17:07.355129 27867 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:07.352917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.22.121.62:39521)
0811 06:17:07.353092 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:07.353097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:07.353509 (+ 412us) server_negotiation.cc:407] Connection header received
0811 06:17:07.353654 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:07.353658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:07.353732 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:07.353816 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:07.354931 (+ 1115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48300: BlockingRecv error: recv got EOF from 127.0.0.1:48300 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:17:07.358351 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27629
W20260811 06:17:07.360831 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27629: Not authorized: Client connection negotiation failed: client connection to 127.22.121.1:35807: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:17:07.361022 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27629
I20260811 06:17:07.379732 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27872
W20260811 06:17:07.382041 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27872: Not authorized: Client connection negotiation failed: client connection to 127.22.121.2:40101: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:17:07.382160 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27872
I20260811 06:17:07.400228 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28131
W20260811 06:17:07.402817 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28131: Not authorized: Client connection negotiation failed: client connection to 127.22.121.3:38463: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:17:07.402943 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28131
I20260811 06:17:07.419831 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27447
I20260811 06:17:07.420837 27556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:07.520671 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27447
2026-08-11T06:17:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3981 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:07 dist-test-slave-gjrz krb5kdc[28400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:07 dist-test-slave-gjrz krb5kdc[28400](info): set up 2 sockets
Aug 11 06:17:07 dist-test-slave-gjrz krb5kdc[28400](info): commencing operation
krb5kdc: starting...
W20260811 06:17:08.735540 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:08 dist-test-slave-gjrz krb5kdc[28400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:08.906443 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46133
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:09.208222 28430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:09.208495 28430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:09.208621 28430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:09.215345 28430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:09.215427 28430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:09.215477 28430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:09.215528 28430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:09.215574 28430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:09.224612 28430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:09.225636 28430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:09.226759 28430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:09.237520 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:09.237550 28442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:09.237803 28435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:09.238327 28430 server_base.cc:1034] running on GCE node
I20260811 06:17:09.239019 28430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:09.240275 28430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:09.241459 28430 hybrid_clock.cc:648] HybridClock initialized: now 1786429029241416 us; error 34 us; skew 500 ppm
I20260811 06:17:09.241760 28430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:09 dist-test-slave-gjrz krb5kdc[28400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429029, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:09.247840 28430 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:09.249161 28430 webserver.cc:460] Webserver started at http://127.22.121.62:33209/ using document root <none> and password file <none>
I20260811 06:17:09.249728 28430 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:09.249820 28430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:09.250052 28430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:09.252731 28430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "861cf922131a48d6bbdadecb3beae37b"
format_stamp: "Formatted at 2026-08-11 06:17:09 on dist-test-slave-gjrz"
server_key: "9ce71534c64e494211fdd08d8d314ba5"
server_key_iv: "7fe2028f91756926578d32b67d007be9"
server_key_version: "encryptionkey@0"
I20260811 06:17:09.253432 28430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "861cf922131a48d6bbdadecb3beae37b"
format_stamp: "Formatted at 2026-08-11 06:17:09 on dist-test-slave-gjrz"
server_key: "9ce71534c64e494211fdd08d8d314ba5"
server_key_iv: "7fe2028f91756926578d32b67d007be9"
server_key_version: "encryptionkey@0"
I20260811 06:17:09.258101 28430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 06:17:09.261189 28455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:09.262339 28430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:09.262478 28430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "861cf922131a48d6bbdadecb3beae37b"
format_stamp: "Formatted at 2026-08-11 06:17:09 on dist-test-slave-gjrz"
server_key: "9ce71534c64e494211fdd08d8d314ba5"
server_key_iv: "7fe2028f91756926578d32b67d007be9"
server_key_version: "encryptionkey@0"
I20260811 06:17:09.262637 28430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:09.307994 28430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:09.311930 28430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:09.312177 28430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:09.328784 28430 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46133
I20260811 06:17:09.328842 28571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46133 every 8 connection(s)
I20260811 06:17:09.329941 28430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:09.330482 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28430
I20260811 06:17:09.330755 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:09.331475 23012 external_mini_cluster.cc:1428] Setting key b6cd3f1eec6463683bd7faa7a71b618f
I20260811 06:17:09.334064 28572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:17:09 dist-test-slave-gjrz krb5kdc[28400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:09.346400 28572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b: Bootstrap starting.
I20260811 06:17:09.349431 28572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:09.350402 28572 log.cc:824] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:09.352535 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:09.333147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.22.121.62:46133)
0811 06:17:09.333869 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:09.333879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:09.333918 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:17:09.335578 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:09.335597 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:09.335928 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:09.336250 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:09.337999 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:09.339364 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:09.341081 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:09.341796 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:09.341936 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:09.346172 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:09.346179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:09.346191 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:09.346223 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:09.348933 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:09.349550 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:09.349556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:09.349561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:09.349727 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:09.350075 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:09.350078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:09.350080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:09.350571 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:09.350839 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:09.351334 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 06:17:09.351682 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":326,"threads_started":1}
I20260811 06:17:09.353025 28572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b: No bootstrap required, opened a new log
I20260811 06:17:09.355916 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:17:09.360038 28572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } }
I20260811 06:17:09.360355 28572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:09.360452 28572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861cf922131a48d6bbdadecb3beae37b, State: Initialized, Role: FOLLOWER
I20260811 06:17:09.360980 28572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } }
I20260811 06:17:09.361198 28572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:09.361323 28572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:09.361480 28572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:09.363461 28572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } }
I20260811 06:17:09.363953 28572 leader_election.cc:302] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861cf922131a48d6bbdadecb3beae37b; no voters:
I20260811 06:17:09.364501 28572 leader_election.cc:288] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:09.364580 28586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:09.365671 28586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [term 1 LEADER]: Becoming Leader. State: Replica: 861cf922131a48d6bbdadecb3beae37b, State: Running, Role: LEADER
I20260811 06:17:09.366184 28586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } }
I20260811 06:17:09.366469 28572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:09.368724 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861cf922131a48d6bbdadecb3beae37b. Latest consensus state: current_term: 1 leader_uuid: "861cf922131a48d6bbdadecb3beae37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } } }
I20260811 06:17:09.369081 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:09.369629 28587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "861cf922131a48d6bbdadecb3beae37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861cf922131a48d6bbdadecb3beae37b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46133 } } }
I20260811 06:17:09.369911 28587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:09.372319 28601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:09.377198 28601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:09.385703 28601 catalog_manager.cc:1347] Generated new cluster ID: 5297d74188a64de8b1a684fcece0879e
I20260811 06:17:09.385839 28601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:09.404610 28601 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:09.406075 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:09.396611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.22.121.62:46133)
0811 06:17:09.396771 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:09.396778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:09.397375 (+ 597us) server_negotiation.cc:407] Connection header received
0811 06:17:09.397563 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:09.397569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:09.397650 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:09.397742 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:09.398952 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:09.400063 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:09.401937 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:09.402609 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:09.402728 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:09.404270 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:09.404276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:09.404280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:09.405246 (+ 966us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:09.405322 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:09.405361 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:09.405469 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:09.405619 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:17:09.405707 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:17:09.406260 28601 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:09.419531 28601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861cf922131a48d6bbdadecb3beae37b: Generated new TSK 0
I20260811 06:17:09.420198 28601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:17:09.431406 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28430
I20260811 06:17:09.432549 28563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:09.522958 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28430
2026-08-11T06:17:09Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1996 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:09 dist-test-slave-gjrz krb5kdc[28650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:09 dist-test-slave-gjrz krb5kdc[28650](info): set up 2 sockets
Aug 11 06:17:09 dist-test-slave-gjrz krb5kdc[28650](info): commencing operation
krb5kdc: starting...
W20260811 06:17:10.726023 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:10 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:10.898048 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41813
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:41813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:11.202135 28669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:11.202390 28669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:11.202483 28669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:11.209501 28669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:11.209592 28669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:11.209645 28669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:11.209712 28669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:11.209772 28669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:11.218899 28669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:41813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:11.220047 28669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:11.221215 28669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:11.232105 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:11.232192 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:11.232151 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:11.232899 28669 server_base.cc:1034] running on GCE node
I20260811 06:17:11.233551 28669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:11.234762 28669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:11.235953 28669 hybrid_clock.cc:648] HybridClock initialized: now 1786429031235902 us; error 39 us; skew 500 ppm
I20260811 06:17:11.236229 28669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:11 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429031, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:11.239187 28669 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:11.240581 28669 webserver.cc:460] Webserver started at http://127.22.121.62:44651/ using document root <none> and password file <none>
I20260811 06:17:11.241176 28669 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:11.241289 28669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:11.241600 28669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:11.244473 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "8557d8565cb24a5be5014dee46d52fee"
server_key_iv: "4f37017c2434b2f7d4f244156184945a"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.245198 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "8557d8565cb24a5be5014dee46d52fee"
server_key_iv: "4f37017c2434b2f7d4f244156184945a"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.249929 28669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:17:11.253083 28693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:11.254235 28669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:11.254391 28669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "8557d8565cb24a5be5014dee46d52fee"
server_key_iv: "4f37017c2434b2f7d4f244156184945a"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.254556 28669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:11.278270 28669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:11.281801 28669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:11.282056 28669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:11.298143 28669 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:41813
I20260811 06:17:11.298180 28808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:41813 every 8 connection(s)
I20260811 06:17:11.299355 28669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:11.301146 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28669
I20260811 06:17:11.301339 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:11.301936 23012 external_mini_cluster.cc:1428] Setting key af7df27c76986071cf2b67c46cff05c4
I20260811 06:17:11.302948 28810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:17:11 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:11.315930 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2: Bootstrap starting.
I20260811 06:17:11.319698 28810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:11.320888 28810 log.cc:824] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:11.323655 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2: No bootstrap required, opened a new log
I20260811 06:17:11.323836 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:11.303734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.22.121.62:41813)
0811 06:17:11.304537 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:11.304552 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:11.304600 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:17:11.306298 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:11.306324 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:11.306781 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:11.307201 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:11.308427 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.310406 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.311591 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.312586 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.312762 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:11.316294 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:11.316305 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:11.316321 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:11.316363 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:11.319972 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:11.320623 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:11.320632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:11.320640 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:11.320837 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:11.321196 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:11.321201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:11.321205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:11.321788 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:11.322133 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:11.322529 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:17:11.322895 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":332,"threads_started":1}
I20260811 06:17:11.331842 28810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } }
I20260811 06:17:11.332162 28810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:11.332252 28810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bdb2c39c3cc444585a1fe2a6923d5f2, State: Initialized, Role: FOLLOWER
I20260811 06:17:11.332753 28810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } }
I20260811 06:17:11.332962 28810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:11.333089 28810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:11.333245 28810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:11.335345 28810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } }
I20260811 06:17:11.335813 28810 leader_election.cc:302] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bdb2c39c3cc444585a1fe2a6923d5f2; no voters:
I20260811 06:17:11.336326 28810 leader_election.cc:288] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:11.336434 28818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:11.337051 28818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5bdb2c39c3cc444585a1fe2a6923d5f2, State: Running, Role: LEADER
I20260811 06:17:11.337617 28818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } }
I20260811 06:17:11.338238 28810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:11.340317 28819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } } }
I20260811 06:17:11.340392 28820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bdb2c39c3cc444585a1fe2a6923d5f2. Latest consensus state: current_term: 1 leader_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdb2c39c3cc444585a1fe2a6923d5f2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41813 } } }
I20260811 06:17:11.340673 28819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:11.340756 28820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:11.342280 28831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:11.347359 28831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:11.355633 28831 catalog_manager.cc:1347] Generated new cluster ID: 7b3d868c57bb468c80856ea4ad6d7c05
I20260811 06:17:11.355732 28831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:11.376155 28831 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:11.377316 28831 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:11.385742 28831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bdb2c39c3cc444585a1fe2a6923d5f2: Generated new TSK 0
I20260811 06:17:11.386379 28831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:11.440816 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:11.807345 28852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:11.807598 28852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:11.807696 28852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:11.814523 28852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:11.814618 28852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:11.814729 28852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:11.823879 28852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:11.824930 28852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:11.826141 28852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:11.838245 28862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:11.838284 28859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:11.838331 28865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:11.838802 28852 server_base.cc:1034] running on GCE node
I20260811 06:17:11.839418 28852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:11.840245 28852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:11.841436 28852 hybrid_clock.cc:648] HybridClock initialized: now 1786429031841385 us; error 44 us; skew 500 ppm
I20260811 06:17:11.841723 28852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:11 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429031, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:11.844743 28852 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:11.846065 28852 webserver.cc:460] Webserver started at http://127.22.121.1:41719/ using document root <none> and password file <none>
I20260811 06:17:11.846705 28852 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:11.846823 28852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:11.847097 28852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:11.849888 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "53570b019cd644adbd88a995296413f7"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "f6d2f2fab595afbdbc8b2ec19cd07ae4"
server_key_iv: "c894d0cf8a35362dd66f45a2cfe396bc"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.850612 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "53570b019cd644adbd88a995296413f7"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "f6d2f2fab595afbdbc8b2ec19cd07ae4"
server_key_iv: "c894d0cf8a35362dd66f45a2cfe396bc"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.855134 28852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 06:17:11.858281 28874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:11.859583 28852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:11.859743 28852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "53570b019cd644adbd88a995296413f7"
format_stamp: "Formatted at 2026-08-11 06:17:11 on dist-test-slave-gjrz"
server_key: "f6d2f2fab595afbdbc8b2ec19cd07ae4"
server_key_iv: "c894d0cf8a35362dd66f45a2cfe396bc"
server_key_version: "encryptionkey@0"
I20260811 06:17:11.859901 28852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:11.879791 28852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:11.883106 28852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:11.883366 28852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:11.884177 28852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:11.885385 28852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:11.885479 28852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:11.885594 28852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:11.885645 28852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:11.913921 28852 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:42115
I20260811 06:17:11.913981 29121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:42115 every 8 connection(s)
I20260811 06:17:11.915107 28852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:11.920331 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28852
I20260811 06:17:11.920567 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:11.921360 23012 external_mini_cluster.cc:1428] Setting key dcf8d8d09fbf859796a104ebb6fa50ce
Aug 11 06:17:11 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429031, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:11.935362 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:11.917460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:39321 (local address 127.22.121.62:41813)
0811 06:17:11.917879 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:11.917883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:11.918662 (+ 779us) server_negotiation.cc:407] Connection header received
0811 06:17:11.920620 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:11.920626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:11.920723 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:11.920819 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:11.922734 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.923733 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.925850 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.926224 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.926310 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:11.930056 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:11.930060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:11.930064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:11.930102 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:11.932478 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:11.933144 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:11.933148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:11.933150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:11.933272 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:11.933671 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:11.933676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:11.933679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:11.933933 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:11.934066 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:11.934729 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 06:17:11.934959 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":322,"thread_start_us":224,"threads_started":1}
I20260811 06:17:11.936189 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:11.917831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41813 (local address 127.22.121.1:39321)
0811 06:17:11.918499 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:11.918535 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:11.920320 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:11.921105 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:11.921118 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:11.921813 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:11.922573 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:11.922594 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.923877 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.923883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:11.925722 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:11.925738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:11.926432 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:11.926438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:11.926703 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:11.927350 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:11.927384 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:11.929851 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:11.932612 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:11.932621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:11.932636 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:11.933024 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:11.933375 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:11.933381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:11.933384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:11.933556 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:11.934056 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:11.934063 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:11.934364 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 06:17:11.934614 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:17:11.935078 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":257,"threads_started":1}
I20260811 06:17:11.937731 29123 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41813
I20260811 06:17:11.937989 29123 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:11.938663 29123 heartbeater.cc:511] Master 127.22.121.62:41813 requested a full tablet report, sending...
I20260811 06:17:11.940933 28727 ts_manager.cc:194] Registered new tserver with Master: 53570b019cd644adbd88a995296413f7 (127.22.121.1:42115)
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:17:11.942569 28727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:39321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:11.964483 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:12.267994 29131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:12.268244 29131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:12.268344 29131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:12.275158 29131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:12.275243 29131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:12.275386 29131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:12.284410 29131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:12.285569 29131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:12.286792 29131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:12.299011 29140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:12.299021 29147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:12.299512 29152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:12.299927 29131 server_base.cc:1034] running on GCE node
I20260811 06:17:12.300518 29131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:12.301509 29131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:12.302740 29131 hybrid_clock.cc:648] HybridClock initialized: now 1786429032302682 us; error 49 us; skew 500 ppm
I20260811 06:17:12.303077 29131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:12.306694 29131 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:12.308212 29131 webserver.cc:460] Webserver started at http://127.22.121.2:41669/ using document root <none> and password file <none>
I20260811 06:17:12.308955 29131 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:12.309074 29131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:12.309396 29131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:12.313089 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "e049483172d743fab70d0964a1876c64"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "b98b16f627c914368997f7dcef784b84"
server_key_iv: "1440f1786a538d20ce8c0a24d615e78d"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.314011 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "e049483172d743fab70d0964a1876c64"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "b98b16f627c914368997f7dcef784b84"
server_key_iv: "1440f1786a538d20ce8c0a24d615e78d"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.319475 29131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 06:17:12.322566 29163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.323776 29131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:12.323912 29131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e049483172d743fab70d0964a1876c64"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "b98b16f627c914368997f7dcef784b84"
server_key_iv: "1440f1786a538d20ce8c0a24d615e78d"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.324070 29131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:12.343823 29131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:12.347472 29131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:12.347719 29131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:12.348615 29131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:12.349821 29131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:12.349895 29131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.350006 29131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:12.350056 29131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.383149 29131 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:32803
I20260811 06:17:12.383189 29399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:32803 every 8 connection(s)
I20260811 06:17:12.384294 29131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:12.389246 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29131
I20260811 06:17:12.389412 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:12.389912 23012 external_mini_cluster.cc:1428] Setting key 93a13cdc0de33e1ca3bdddf6c55261ae
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:12.403417 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.386487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:46921 (local address 127.22.121.62:41813)
0811 06:17:12.386635 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:12.386639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:12.387775 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 06:17:12.389561 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:12.389567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:12.389653 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:12.389758 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:12.391204 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.392158 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.393616 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.394187 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.394294 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.398127 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:12.398132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:12.398135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:12.398167 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:12.400664 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.401311 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.401315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.401317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.401444 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.401825 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.401830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.401833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.402092 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:12.402223 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:12.402831 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 06:17:12.403057 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:17:12.404218 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.386867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41813 (local address 127.22.121.2:46921)
0811 06:17:12.387621 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:12.387665 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:12.389276 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:12.389927 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:12.389934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:12.390421 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:12.391022 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:12.391045 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.392303 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.392307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:12.393470 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:12.393482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.394538 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.394557 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:12.394816 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.395457 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:12.395486 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:12.397917 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:12.400787 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:12.400796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:12.400810 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:12.401201 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:12.401543 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:12.401548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:12.401551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:12.401722 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:12.402211 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:12.402219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:12.402497 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 06:17:12.402720 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 06:17:12.403194 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":324,"threads_started":1}
I20260811 06:17:12.405647 29400 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41813
I20260811 06:17:12.405910 29400 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:12.406538 29400 heartbeater.cc:511] Master 127.22.121.62:41813 requested a full tablet report, sending...
I20260811 06:17:12.408007 28727 ts_manager.cc:194] Registered new tserver with Master: e049483172d743fab70d0964a1876c64 (127.22.121.2:32803)
I20260811 06:17:12.409152 28727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:46921
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:12.432538 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:12.733462 29410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:12.733713 29410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:12.733851 29410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:12.740811 29410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:12.740916 29410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:12.741053 29410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:12.750473 29410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:12.751664 29410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:12.752910 29410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:12.764871 29423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:12.764999 29422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:12.765340 29425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:12.765905 29410 server_base.cc:1034] running on GCE node
I20260811 06:17:12.766484 29410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:12.767346 29410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:12.768540 29410 hybrid_clock.cc:648] HybridClock initialized: now 1786429032768492 us; error 59 us; skew 500 ppm
I20260811 06:17:12.768813 29410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:12.776082 29410 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:12.777443 29410 webserver.cc:460] Webserver started at http://127.22.121.3:40781/ using document root <none> and password file <none>
I20260811 06:17:12.778050 29410 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:12.778149 29410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:12.778401 29410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:12.781235 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "74ebf56844834dd9ae8ed31dc9ff547c"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "bbc97933b6cf52797f931c1ded7baaa3"
server_key_iv: "48f422b9a0e87dc04e6b8cd82d6c2896"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.781941 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "74ebf56844834dd9ae8ed31dc9ff547c"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "bbc97933b6cf52797f931c1ded7baaa3"
server_key_iv: "48f422b9a0e87dc04e6b8cd82d6c2896"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.786655 29410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:17:12.789768 29447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.791029 29410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:12.791183 29410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "74ebf56844834dd9ae8ed31dc9ff547c"
format_stamp: "Formatted at 2026-08-11 06:17:12 on dist-test-slave-gjrz"
server_key: "bbc97933b6cf52797f931c1ded7baaa3"
server_key_iv: "48f422b9a0e87dc04e6b8cd82d6c2896"
server_key_version: "encryptionkey@0"
I20260811 06:17:12.791384 29410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:12.820788 29410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:12.824739 29410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:12.825012 29410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:12.825863 29410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:12.827157 29410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:12.827227 29410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.827358 29410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:12.827415 29410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:12.856778 29410 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41389
I20260811 06:17:12.856837 29673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41389 every 8 connection(s)
I20260811 06:17:12.858034 29410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:12.866703 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29410
I20260811 06:17:12.866879 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:12.867317 23012 external_mini_cluster.cc:1428] Setting key 91e353199ce5785355b93637c7518089
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:12.876577 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.860487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:56981 (local address 127.22.121.62:41813)
0811 06:17:12.860652 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:12.860657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:12.861699 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 06:17:12.863526 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:12.863531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:12.863601 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:12.863684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:12.865135 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.865857 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.867428 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.867881 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.868016 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.871436 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:12.871441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:12.871444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:12.871486 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:12.873714 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.874392 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.874396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.874398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.874533 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.874933 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.874936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.874938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.875193 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:12.875358 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:12.876014 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 06:17:12.876226 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:17:12.877398 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.860912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41813 (local address 127.22.121.3:56981)
0811 06:17:12.861556 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:12.861592 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:12.863286 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:12.863891 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:12.863924 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:12.864413 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:12.864993 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:12.865011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.865995 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.865999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:12.867292 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:12.867307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.868104 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.868112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:12.868381 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.868992 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:12.869017 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:12.871218 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:12.873849 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:12.873858 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:12.873873 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:12.874272 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:12.874637 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:12.874642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:12.874646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:12.874821 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:12.875364 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:12.875372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:12.875662 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 06:17:12.875877 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 06:17:12.876358 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":257,"threads_started":1}
I20260811 06:17:12.878538 29674 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41813
I20260811 06:17:12.878796 29674 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:12.879478 29674 heartbeater.cc:511] Master 127.22.121.62:41813 requested a full tablet report, sending...
I20260811 06:17:12.880762 28727 ts_manager.cc:194] Registered new tserver with Master: 74ebf56844834dd9ae8ed31dc9ff547c (127.22.121.3:41389)
I20260811 06:17:12.881616 28727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:56981
I20260811 06:17:12.882464 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:12.935817 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.917622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.22.121.62:41813)
0811 06:17:12.917857 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:12.917865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:12.919858 (+ 1993us) server_negotiation.cc:407] Connection header received
0811 06:17:12.920070 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:12.920084 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:12.920164 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:12.920264 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:12.921818 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.922596 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.924150 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.924807 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.924936 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.929912 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:12.929920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:12.929924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:12.929964 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:12.933188 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.933788 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.933795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.933799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.933957 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.934324 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.934329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.934333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.934667 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:12.934899 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:12.935185 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:17:12.935441 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 06:17:12.936380 28725 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59654
I20260811 06:17:12.940253 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28852
I20260811 06:17:12.945413 29123 heartbeater.cc:503] Master 127.22.121.62:41813 was elected leader, sending a full tablet report...
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:12.959941 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.942231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.22.121.1:42115)
0811 06:17:12.942800 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:12.942807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:12.942832 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:17:12.942886 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:12.942892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:12.943091 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:12.943246 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:12.944493 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.946604 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.948067 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.948920 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.949049 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.953355 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:12.953366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:12.953380 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:12.953411 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:12.956961 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.957814 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.957823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.957826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.957983 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.958366 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.958371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.958375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.958713 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:12.959030 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:12.959335 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:17:12.959584 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":282,"threads_started":1}
W20260811 06:17:12.961300 29114 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44032
W20260811 06:17:12.962631 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28852: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:12.962844 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28852
I20260811 06:17:12.985369 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29131
Aug 11 06:17:12 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:13.001477 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:12.986207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.22.121.2:32803)
0811 06:17:12.986736 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:12.986743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:12.986769 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:17:12.986828 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:12.986834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:12.987044 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:12.987168 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:12.988360 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.989496 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.990840 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:12.991453 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:12.991527 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:12.995535 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:12.995546 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:12.995560 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:12.995605 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:12.998834 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:12.999720 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:12.999730 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:12.999733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:12.999898 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:13.000226 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:13.000229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:13.000231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:13.000553 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:13.000784 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:13.000990 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 06:17:13.001195 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":246,"threads_started":1}
W20260811 06:17:13.002677 29393 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46388
W20260811 06:17:13.003698 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29131: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:13.003844 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29131
I20260811 06:17:13.027957 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29410
Aug 11 06:17:13 dist-test-slave-gjrz krb5kdc[28650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429032, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:13.044085 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:13.029005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.22.121.3:41389)
0811 06:17:13.029701 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:13.029710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:13.029746 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:17:13.029817 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:13.029826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:13.030065 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:13.030202 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:13.031494 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:13.032553 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:13.033762 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:13.034309 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:13.034411 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:13.038597 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:13.038608 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:13.038623 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:13.038655 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:13.041254 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:13.041860 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:13.041867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:13.041869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:13.042017 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:13.042378 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:13.042383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:13.042385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:13.042758 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:13.043091 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:13.043547 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 06:17:13.043812 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":351,"threads_started":1}
W20260811 06:17:13.045293 29659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48960
W20260811 06:17:13.046336 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29410: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:17:13.046469 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29410
I20260811 06:17:13.071775 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28669
I20260811 06:17:13.072831 28801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:13.172474 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28669
2026-08-11T06:17:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3651 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:13 dist-test-slave-gjrz krb5kdc[29714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:13 dist-test-slave-gjrz krb5kdc[29714](info): set up 2 sockets
Aug 11 06:17:13 dist-test-slave-gjrz krb5kdc[29714](info): commencing operation
krb5kdc: starting...
W20260811 06:17:14.402942 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:14 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:14.572386 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35585
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:14.871661 29739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:14.871906 29739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:14.872035 29739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:14.878819 29739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:14.878909 29739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:14.878983 29739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:14.879057 29739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:14.879112 29739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:14.888257 29739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:35585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:14.889361 29739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:14.890550 29739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:14.901387 29755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:14.901360 29753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:14.901576 29751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:14.902329 29739 server_base.cc:1034] running on GCE node
I20260811 06:17:14.903021 29739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:14.904305 29739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:14.905504 29739 hybrid_clock.cc:648] HybridClock initialized: now 1786429034905449 us; error 37 us; skew 500 ppm
I20260811 06:17:14.905817 29739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:14 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429034, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:14.912442 29739 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:14.913784 29739 webserver.cc:460] Webserver started at http://127.22.121.62:32831/ using document root <none> and password file <none>
I20260811 06:17:14.914369 29739 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:14.914472 29739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:14.914724 29739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:14.917560 29739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "4aa4c618c5134d9ab304f5b6f5f9717f"
format_stamp: "Formatted at 2026-08-11 06:17:14 on dist-test-slave-gjrz"
server_key: "1603cddbcc86ec99583e484522278f97"
server_key_iv: "223282716c0a04c8cdf3f5edd2d23eec"
server_key_version: "encryptionkey@0"
I20260811 06:17:14.918360 29739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "4aa4c618c5134d9ab304f5b6f5f9717f"
format_stamp: "Formatted at 2026-08-11 06:17:14 on dist-test-slave-gjrz"
server_key: "1603cddbcc86ec99583e484522278f97"
server_key_iv: "223282716c0a04c8cdf3f5edd2d23eec"
server_key_version: "encryptionkey@0"
I20260811 06:17:14.923403 29739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:17:14.926549 29764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:14.927757 29739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:14.927899 29739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "4aa4c618c5134d9ab304f5b6f5f9717f"
format_stamp: "Formatted at 2026-08-11 06:17:14 on dist-test-slave-gjrz"
server_key: "1603cddbcc86ec99583e484522278f97"
server_key_iv: "223282716c0a04c8cdf3f5edd2d23eec"
server_key_version: "encryptionkey@0"
I20260811 06:17:14.928072 29739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:14.946162 29739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:14.949556 29739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:14.949788 29739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:14.966053 29739 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:35585
I20260811 06:17:14.966074 29862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:35585 every 8 connection(s)
I20260811 06:17:14.967228 29739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:14.970548 29863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:14.974769 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29739
I20260811 06:17:14.974972 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:14.975868 23012 external_mini_cluster.cc:1428] Setting key 3c29e7f1e6acc6b37214626f080da5bd
I20260811 06:17:14.981856 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f: Bootstrap starting.
I20260811 06:17:14.985296 29863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:14.986441 29863 log.cc:824] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:14.989449 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f: No bootstrap required, opened a new log
Aug 11 06:17:14 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:14.998018 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:14.977757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.22.121.62:35585)
0811 06:17:14.978511 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:14.978522 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:14.978564 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:17:14.980379 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:14.980408 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:14.980794 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:14.981101 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:14.982933 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:14.984309 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:14.985966 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:14.986657 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:14.986776 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:14.991224 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:14.991231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:14.991245 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:14.991310 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:14.994282 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:14.995055 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:14.995061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:14.995067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:14.995219 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:14.995597 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:14.995600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:14.995602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:14.996124 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:14.996332 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:14.996753 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 06:17:14.997086 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":360,"threads_started":1}
I20260811 06:17:14.997944 29863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } }
I20260811 06:17:14.998303 29863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:14.998414 29863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa4c618c5134d9ab304f5b6f5f9717f, State: Initialized, Role: FOLLOWER
I20260811 06:17:14.998934 29863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } }
I20260811 06:17:14.999153 29863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:14.999253 29863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:14.999480 29863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:15.001581 29863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } }
I20260811 06:17:15.002041 29863 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa4c618c5134d9ab304f5b6f5f9717f; no voters:
I20260811 06:17:15.002650 29863 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:15.002743 29881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:15.003942 29881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [term 1 LEADER]: Becoming Leader. State: Replica: 4aa4c618c5134d9ab304f5b6f5f9717f, State: Running, Role: LEADER
I20260811 06:17:15.004472 29881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } }
I20260811 06:17:15.004778 29863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:15.007198 29885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa4c618c5134d9ab304f5b6f5f9717f. Latest consensus state: current_term: 1 leader_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } } }
I20260811 06:17:15.007675 29885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:15.007537 29884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa4c618c5134d9ab304f5b6f5f9717f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35585 } } }
I20260811 06:17:15.007858 29884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:15.010864 29900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:15.016264 29900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:15.024580 29900 catalog_manager.cc:1347] Generated new cluster ID: 39d42807a8f64d0c8302feea61ba1b40
I20260811 06:17:15.024673 29900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:15.038498 29900 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:15.040187 29900 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:15.055567 29900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa4c618c5134d9ab304f5b6f5f9717f: Generated new TSK 0
I20260811 06:17:15.056464 29900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:15.108419 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:15.415529 29921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:15.415772 29921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:15.415889 29921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:15.422695 29921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:15.422796 29921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:15.422931 29921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:15.432359 29921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:15.433449 29921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:15.434687 29921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:15.446574 29926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:15.446619 29931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:15.446651 29928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:15.447407 29921 server_base.cc:1034] running on GCE node
I20260811 06:17:15.447892 29921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:15.448665 29921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:15.449848 29921 hybrid_clock.cc:648] HybridClock initialized: now 1786429035449794 us; error 48 us; skew 500 ppm
I20260811 06:17:15.450120 29921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:15 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429035, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:15.453068 29921 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:15.454388 29921 webserver.cc:460] Webserver started at http://127.22.121.1:39117/ using document root <none> and password file <none>
I20260811 06:17:15.454983 29921 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:15.455098 29921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:15.455412 29921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:15.458084 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "6e0b8cd2144d4bb99f72cd1df45dfbe8"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "ffc4cda849db99ac06be57611517a50d"
server_key_iv: "56f2826be8334592b327f40ea5a449f0"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.458832 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "6e0b8cd2144d4bb99f72cd1df45dfbe8"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "ffc4cda849db99ac06be57611517a50d"
server_key_iv: "56f2826be8334592b327f40ea5a449f0"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.463755 29921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:17:15.466809 29947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.467985 29921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:15.468122 29921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "6e0b8cd2144d4bb99f72cd1df45dfbe8"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "ffc4cda849db99ac06be57611517a50d"
server_key_iv: "56f2826be8334592b327f40ea5a449f0"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.468287 29921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:15.478452 29921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:15.481812 29921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:15.482061 29921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:15.482868 29921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:15.484086 29921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:15.484161 29921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.484278 29921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:15.484328 29921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.515092 29921 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:41629
I20260811 06:17:15.515136 30173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:41629 every 8 connection(s)
I20260811 06:17:15.516219 29921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:15.525120 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29921
I20260811 06:17:15.525319 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:15.525964 23012 external_mini_cluster.cc:1428] Setting key d5eee78263f1b3862c947d4b3f3d8f27
Aug 11 06:17:15 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429035, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:15.535161 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:15.518404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42249 (local address 127.22.121.62:35585)
0811 06:17:15.518802 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:15.518807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:15.519645 (+ 838us) server_negotiation.cc:407] Connection header received
0811 06:17:15.521350 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:15.521356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:15.521429 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:15.521525 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:15.522949 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.523702 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.525477 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.525903 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.526034 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:15.529714 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:15.529719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:15.529722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:15.529754 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:15.532290 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:15.533038 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:15.533042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:15.533044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:15.533152 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:15.533569 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:15.533572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:15.533574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:15.533829 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:15.533937 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:15.534617 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 06:17:15.534785 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
I20260811 06:17:15.535969 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:15.518803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35585 (local address 127.22.121.1:42249)
0811 06:17:15.519493 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:15.519534 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:15.521092 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:15.521759 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:15.521766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:15.522244 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:15.522782 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:15.522800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.523851 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.523855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:15.525325 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:15.525347 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.526153 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.526161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:15.526432 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:15.527074 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:15.527097 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:15.529496 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:15.532414 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:15.532422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:15.532436 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:15.532944 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:15.533243 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:15.533247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:15.533250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:15.533465 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:15.533940 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:15.533947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:15.534297 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 06:17:15.534559 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 06:17:15.534973 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":258,"threads_started":1}
I20260811 06:17:15.537272 30175 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35585
I20260811 06:17:15.537567 30175 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:15.538335 30175 heartbeater.cc:511] Master 127.22.121.62:35585 requested a full tablet report, sending...
I20260811 06:17:15.540395 29793 ts_manager.cc:194] Registered new tserver with Master: 6e0b8cd2144d4bb99f72cd1df45dfbe8 (127.22.121.1:41629)
I20260811 06:17:15.541797 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:42249
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:15.568593 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:15.870178 30186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:15.870446 30186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:15.870579 30186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:15.877564 30186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:15.877668 30186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:15.877811 30186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:15.886804 30186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:15.887871 30186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:15.889035 30186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:15.901157 30197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:15.901108 30200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:15.901192 30207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:15.901768 30186 server_base.cc:1034] running on GCE node
I20260811 06:17:15.902331 30186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:15.903115 30186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:15.904296 30186 hybrid_clock.cc:648] HybridClock initialized: now 1786429035904248 us; error 39 us; skew 500 ppm
I20260811 06:17:15.904582 30186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:15 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429035, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:15.907406 30186 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:15.908658 30186 webserver.cc:460] Webserver started at http://127.22.121.2:46131/ using document root <none> and password file <none>
I20260811 06:17:15.909258 30186 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:15.909371 30186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:15.909653 30186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:15.912415 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "358d82c0a6ab49d3b5e418e5f569f922"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "33683e7fb579a94d09d6ab8491581db6"
server_key_iv: "bda36fedcd2115ac4e66173fe8771897"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.913121 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "358d82c0a6ab49d3b5e418e5f569f922"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "33683e7fb579a94d09d6ab8491581db6"
server_key_iv: "bda36fedcd2115ac4e66173fe8771897"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.917889 30186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:17:15.921017 30217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.922204 30186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:15.922348 30186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "358d82c0a6ab49d3b5e418e5f569f922"
format_stamp: "Formatted at 2026-08-11 06:17:15 on dist-test-slave-gjrz"
server_key: "33683e7fb579a94d09d6ab8491581db6"
server_key_iv: "bda36fedcd2115ac4e66173fe8771897"
server_key_version: "encryptionkey@0"
I20260811 06:17:15.922539 30186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:15.934059 30186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:15.937429 30186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:15.937680 30186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:15.938495 30186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:15.939697 30186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:15.939793 30186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.939905 30186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:15.939958 30186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:15.970057 30186 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:35397
I20260811 06:17:15.970089 30416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:35397 every 8 connection(s)
I20260811 06:17:15.971364 30186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:15.972545 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30186
I20260811 06:17:15.972715 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:15.973318 23012 external_mini_cluster.cc:1428] Setting key 194214559f53836723fc81aebb72379c
Aug 11 06:17:15 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429035, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:15.993541 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:15.974076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:50333 (local address 127.22.121.62:35585)
0811 06:17:15.974963 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:15.974970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:15.975643 (+ 673us) server_negotiation.cc:407] Connection header received
0811 06:17:15.978125 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:15.978131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:15.978209 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:15.978322 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:15.980194 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.981301 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.983402 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.983988 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.984126 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:15.987592 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:15.987600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:15.987604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:15.987644 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:15.990600 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:15.991295 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:15.991300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:15.991303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:15.991450 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:15.991913 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:15.991917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:15.991921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:15.992254 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:15.992416 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:15.992977 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 06:17:15.993203 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759}
I20260811 06:17:15.994182 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:15.974542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35585 (local address 127.22.121.2:50333)
0811 06:17:15.975492 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:15.975544 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:15.977811 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:15.978558 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:15.978569 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:15.979251 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:15.980009 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:15.980034 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.981469 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.981474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:15.983235 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:15.983256 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:15.984225 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:15.984230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:15.984424 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:15.985047 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:15.985068 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:15.987353 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:15.990722 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:15.990729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:15.990740 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:15.991143 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:15.991558 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:15.991563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:15.991566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:15.991797 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:15.992361 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:15.992366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:15.992634 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 06:17:15.992838 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 06:17:15.993193 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":386,"threads_started":1}
I20260811 06:17:15.995529 30417 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35585
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
I20260811 06:17:15.995878 30417 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:15.996577 30417 heartbeater.cc:511] Master 127.22.121.62:35585 requested a full tablet report, sending...
I20260811 06:17:15.997875 29793 ts_manager.cc:194] Registered new tserver with Master: 358d82c0a6ab49d3b5e418e5f569f922 (127.22.121.2:35397)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
I20260811 06:17:15.998819 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:50333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:16.021944 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:16.330379 30428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:16.330641 30428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:16.330783 30428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:16.337783 30428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:16.337893 30428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:16.338047 30428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:16.347246 30428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:16.348502 30428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:16.349783 30428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:16.362018 30439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:16.362062 30441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:16.362247 30446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:16.363170 30428 server_base.cc:1034] running on GCE node
I20260811 06:17:16.363754 30428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:16.364615 30428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:16.365815 30428 hybrid_clock.cc:648] HybridClock initialized: now 1786429036365766 us; error 37 us; skew 500 ppm
I20260811 06:17:16.366110 30428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:16.369065 30428 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:16.370393 30428 webserver.cc:460] Webserver started at http://127.22.121.3:46223/ using document root <none> and password file <none>
I20260811 06:17:16.371013 30428 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:16.371119 30428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:16.371418 30428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:16.374145 30428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "1616bb5bed6240dda35175e520fe0724"
format_stamp: "Formatted at 2026-08-11 06:17:16 on dist-test-slave-gjrz"
server_key: "85bcf3d41b821935cd9fbd84e19fab95"
server_key_iv: "d755b6947963fdd7650b9d8f4ac44bb7"
server_key_version: "encryptionkey@0"
I20260811 06:17:16.374842 30428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "1616bb5bed6240dda35175e520fe0724"
format_stamp: "Formatted at 2026-08-11 06:17:16 on dist-test-slave-gjrz"
server_key: "85bcf3d41b821935cd9fbd84e19fab95"
server_key_iv: "d755b6947963fdd7650b9d8f4ac44bb7"
server_key_version: "encryptionkey@0"
I20260811 06:17:16.379563 30428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 06:17:16.382710 30457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:16.383951 30428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:16.384093 30428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1616bb5bed6240dda35175e520fe0724"
format_stamp: "Formatted at 2026-08-11 06:17:16 on dist-test-slave-gjrz"
server_key: "85bcf3d41b821935cd9fbd84e19fab95"
server_key_iv: "d755b6947963fdd7650b9d8f4ac44bb7"
server_key_version: "encryptionkey@0"
I20260811 06:17:16.384270 30428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:16.398284 30428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:16.401688 30428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:16.401953 30428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:16.402792 30428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:16.404018 30428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:16.404098 30428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:16.404206 30428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:16.404254 30428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:16.433784 30428 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42339
I20260811 06:17:16.433843 30659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42339 every 8 connection(s)
I20260811 06:17:16.434948 30428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:16.437660 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30428
I20260811 06:17:16.437804 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:16.438223 23012 external_mini_cluster.cc:1428] Setting key af96d9fe31a8331fe7b597aecbb581bf
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:16.453101 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.437176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:46815 (local address 127.22.121.62:35585)
0811 06:17:16.437381 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.437387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.438599 (+ 1212us) server_negotiation.cc:407] Connection header received
0811 06:17:16.440664 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.440669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.440750 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.440833 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.442223 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.442950 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.444475 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.444897 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.445008 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.448076 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.448080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.448083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.448116 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.450307 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.450951 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.450955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.450957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.451070 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.451511 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.451514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.451516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.451779 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.451909 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.452547 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.452775 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:17:16.453860 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.437657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35585 (local address 127.22.121.3:46815)
0811 06:17:16.438444 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:16.438488 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:16.440417 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:16.441071 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:16.441079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:16.441540 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:16.442070 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:16.442087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.443099 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.443102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:16.444334 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:16.444347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.445102 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.445109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:16.445331 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.445844 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:16.445864 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:16.447847 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:16.450433 (+ 2586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:16.450442 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:16.450456 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:16.450828 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:16.451170 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:16.451175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:16.451179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:16.451399 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:16.451900 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:16.451907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:16.452210 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 06:17:16.452417 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 06:17:16.452861 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":326,"threads_started":1}
I20260811 06:17:16.455020 30661 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35585
I20260811 06:17:16.455296 30661 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:16.456002 30661 heartbeater.cc:511] Master 127.22.121.62:35585 requested a full tablet report, sending...
I20260811 06:17:16.457278 29793 ts_manager.cc:194] Registered new tserver with Master: 1616bb5bed6240dda35175e520fe0724 (127.22.121.3:42339)
I20260811 06:17:16.458076 29793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:46815
I20260811 06:17:16.464674 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:16.511168 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.497453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.22.121.1:41629)
0811 06:17:16.497949 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.497958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.497987 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:17:16.498147 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.498154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.498358 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.498503 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.499944 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.500903 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.502098 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.502618 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.502731 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.506273 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.506284 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.506296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.506333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.508786 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.509332 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.509338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.509340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.509465 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.509793 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.509796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.509798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.510144 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.510375 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.510586 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.510809 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:16.528223 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.515021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.22.121.62:35585)
0811 06:17:16.515218 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.515225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.516143 (+ 918us) server_negotiation.cc:407] Connection header received
0811 06:17:16.516245 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.516250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.516339 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.516443 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.517528 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.518614 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.519822 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.520468 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.520582 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.523894 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.523898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.523902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.523932 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.526088 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.526717 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.526721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.526723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.526839 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.527134 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.527137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.527139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.527459 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.527649 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.527863 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.528016 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 06:17:16.528793 29793 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35728
I20260811 06:17:16.529667 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29921
I20260811 06:17:16.540920 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.530500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.22.121.1:41629)
0811 06:17:16.530671 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.530677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.530831 (+ 154us) server_negotiation.cc:407] Connection header received
0811 06:17:16.530977 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.530983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.531052 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.531135 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.532289 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.532937 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.534068 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.534464 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.534567 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.536519 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.536525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.536528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.536556 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.538633 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.539192 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.539198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.539202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.539366 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.539703 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.539708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.539712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.539940 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.540153 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.540420 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.540622 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:17:16.541456 30163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:16.619903 30175 heartbeater.cc:503] Master 127.22.121.62:35585 was elected leader, sending a full tablet report...
I20260811 06:17:16.620805 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29921
W20260811 06:17:16.641702 29768 connection.cc:535] server connection from 127.22.121.1:42249 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:17:16.641932 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30186
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:16.656584 30682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.642764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.22.121.2:35397)
0811 06:17:16.643328 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.643335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.643362 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:17:16.643420 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.643426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.643638 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.643760 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.644852 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.645836 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.646976 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.647630 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.647733 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.651589 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.651599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.651614 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.651647 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.654174 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.654735 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.654741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.654743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.654857 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.655136 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.655139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.655141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.655580 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.655814 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.656071 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.656304 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":253,"threads_started":1}
I20260811 06:17:16.657858 30403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:16.738505 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30186
I20260811 06:17:16.763013 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30428
Aug 11 06:17:16 dist-test-slave-gjrz krb5kdc[29714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:16.778661 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:16.763926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.22.121.3:42339)
0811 06:17:16.764566 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:16.764574 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:16.764605 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:17:16.764667 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:16.764674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:16.764882 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:16.765033 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:16.766314 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.767465 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.768741 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:16.769303 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:16.769418 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:16.773484 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:16.773495 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:16.773510 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:16.773542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:16.776023 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.776663 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.776670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.776672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.776793 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:16.777141 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:16.777145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:16.777147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:16.777487 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:16.777776 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:16.778109 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:17:16.778356 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":337,"threads_started":1}
I20260811 06:17:16.779902 30653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:16.860340 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30428
I20260811 06:17:16.882588 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29739
I20260811 06:17:16.883767 29847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:16.982999 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29739
2026-08-11T06:17:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3813 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:17 dist-test-slave-gjrz krb5kdc[30696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:17 dist-test-slave-gjrz krb5kdc[30696](info): set up 2 sockets
Aug 11 06:17:17 dist-test-slave-gjrz krb5kdc[30696](info): commencing operation
krb5kdc: starting...
W20260811 06:17:18.207780 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:18 dist-test-slave-gjrz krb5kdc[30696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:18.390152 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46295
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:18.694070 30735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:18.694324 30735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:18.694438 30735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:18.701287 30735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:18.701375 30735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:18.701444 30735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:18.701517 30735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:18.701577 30735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:18.710686 30735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:18.711803 30735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:18.712982 30735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:18.723600 30747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:18.723562 30750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:18.724025 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:18.724562 30735 server_base.cc:1034] running on GCE node
I20260811 06:17:18.725253 30735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:18.726418 30735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:18.727623 30735 hybrid_clock.cc:648] HybridClock initialized: now 1786429038727586 us; error 26 us; skew 500 ppm
I20260811 06:17:18.727974 30735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:18 dist-test-slave-gjrz krb5kdc[30696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:18.730846 30735 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:18.731945 30735 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:18.732048 30735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:18.732352 30735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:18.735114 30735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "2fd3db180c6b42d699eee7289e29d8d4"
format_stamp: "Formatted at 2026-08-11 06:17:18 on dist-test-slave-gjrz"
server_key: "e07b5b384c14cd35a6ed59a7eec18aec"
server_key_iv: "0871c528b1170cb4beff8633db10b183"
server_key_version: "encryptionkey@0"
I20260811 06:17:18.735895 30735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "2fd3db180c6b42d699eee7289e29d8d4"
format_stamp: "Formatted at 2026-08-11 06:17:18 on dist-test-slave-gjrz"
server_key: "e07b5b384c14cd35a6ed59a7eec18aec"
server_key_iv: "0871c528b1170cb4beff8633db10b183"
server_key_version: "encryptionkey@0"
I20260811 06:17:18.740509 30735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 06:17:18.743695 30762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:18.744910 30735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 06:17:18.745105 30735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "2fd3db180c6b42d699eee7289e29d8d4"
format_stamp: "Formatted at 2026-08-11 06:17:18 on dist-test-slave-gjrz"
server_key: "e07b5b384c14cd35a6ed59a7eec18aec"
server_key_iv: "0871c528b1170cb4beff8633db10b183"
server_key_version: "encryptionkey@0"
I20260811 06:17:18.745296 30735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:18.768712 30735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:18.772565 30735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:18.772789 30735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:18.789361 30735 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46295
I20260811 06:17:18.789381 30847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46295 every 8 connection(s)
I20260811 06:17:18.790129 30735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:18.793459 30848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:18.794145 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30735
I20260811 06:17:18.794358 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:18.795145 23012 external_mini_cluster.cc:1428] Setting key ca517112663ee71f8cc7738dc4eba0c6
I20260811 06:17:18.807294 30848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: Bootstrap starting.
I20260811 06:17:18.810222 30848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:18.811334 30848 log.cc:824] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:18.813930 30848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: No bootstrap required, opened a new log
Aug 11 06:17:18 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:18.821219 30848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } }
I20260811 06:17:18.821640 30848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:18.821741 30848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd3db180c6b42d699eee7289e29d8d4, State: Initialized, Role: FOLLOWER
I20260811 06:17:18.822350 30848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } }
I20260811 06:17:18.822569 30848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:18.822706 30848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:18.822902 30848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:18.825307 30848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } }
I20260811 06:17:18.825853 30848 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd3db180c6b42d699eee7289e29d8d4; no voters:
I20260811 06:17:18.826452 30848 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:18.826563 30855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:18.827634 30855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd3db180c6b42d699eee7289e29d8d4, State: Running, Role: LEADER
I20260811 06:17:18.828259 30855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } }
I20260811 06:17:18.828553 30848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:18.830214 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:18.797361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.22.121.62:46295)
0811 06:17:18.798361 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:18.798381 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:18.809917 (+ 11536us) server_negotiation.cc:407] Connection header received
0811 06:17:18.812016 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:18.812044 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:18.812513 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:18.812872 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:18.814368 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:18.815714 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:18.816939 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:18.817607 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:18.817798 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:18.821947 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:18.821956 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:18.821969 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:18.822003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:18.824992 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:18.825743 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:18.825772 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:18.825777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:18.825932 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:18.826376 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:18.826385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:18.826388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:18.827150 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:18.828015 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:18.828393 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 06:17:18.828821 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":525,"threads_started":1}
I20260811 06:17:18.831144 30857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fd3db180c6b42d699eee7289e29d8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } } }
I20260811 06:17:18.831494 30857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:18.833570 30858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd3db180c6b42d699eee7289e29d8d4. Latest consensus state: current_term: 1 leader_uuid: "2fd3db180c6b42d699eee7289e29d8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd3db180c6b42d699eee7289e29d8d4" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46295 } } }
I20260811 06:17:18.833938 30858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:18.834456 30864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:18.839340 30864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:18.847689 30864 catalog_manager.cc:1347] Generated new cluster ID: ff45b58e635f4ab6840813cee348e058
I20260811 06:17:18.847781 30864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:18.866575 30864 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:18.867771 30864 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:18.876741 30864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: Generated new TSK 0
I20260811 06:17:18.877550 30864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:18.930637 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:19.249473 30895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:19.249720 30895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:19.249845 30895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:19.257109 30895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:19.257201 30895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:19.257341 30895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:19.266548 30895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:19.267683 30895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:19.268848 30895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:19.280608 30904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:19.280683 30908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:19.280653 30902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:19.281581 30895 server_base.cc:1034] running on GCE node
I20260811 06:17:19.282160 30895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:19.283033 30895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:19.284236 30895 hybrid_clock.cc:648] HybridClock initialized: now 1786429039284180 us; error 42 us; skew 500 ppm
I20260811 06:17:19.284547 30895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:19 dist-test-slave-gjrz krb5kdc[30696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429039, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:19.287528 30895 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:19.288529 30895 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:19.288626 30895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:19.288908 30895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:19.291664 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "d3cd92a972e445ecb1cabf176685c81e"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "4dbae21fb3c0785c756170ad08b53450"
server_key_iv: "1c1e32c48b540f692e864d416bb15aeb"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.292420 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "d3cd92a972e445ecb1cabf176685c81e"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "4dbae21fb3c0785c756170ad08b53450"
server_key_iv: "1c1e32c48b540f692e864d416bb15aeb"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.297040 30895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:17:19.300163 30918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.301340 30895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:17:19.301483 30895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "d3cd92a972e445ecb1cabf176685c81e"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "4dbae21fb3c0785c756170ad08b53450"
server_key_iv: "1c1e32c48b540f692e864d416bb15aeb"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.301671 30895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:19.320199 30895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:19.323817 30895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:19.324046 30895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:19.324934 30895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:19.326198 30895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:19.326301 30895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.326411 30895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:19.326480 30895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.355069 30895 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:43295
I20260811 06:17:19.355135 31113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:43295 every 8 connection(s)
I20260811 06:17:19.355927 30895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:19.358346 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30895
I20260811 06:17:19.358548 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:19.359223 23012 external_mini_cluster.cc:1428] Setting key 6790c83599ea52765f4b5a87229f1e7a
Aug 11 06:17:19 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429039, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:19.376344 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:19.358193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:35853 (local address 127.22.121.62:46295)
0811 06:17:19.358680 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:19.358686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:19.359704 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 06:17:19.361610 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:19.361614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:19.361675 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:19.361758 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:19.363691 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.364411 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.366431 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.366801 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.366897 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:19.370414 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:19.370420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:19.370425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:19.370460 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:19.373431 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:19.374111 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:19.374116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:19.374120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:19.374252 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:19.374628 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:19.374633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:19.374637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:19.374962 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:19.375105 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:19.375743 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 06:17:19.375978 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":233,"threads_started":1}
I20260811 06:17:19.377195 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:19.358736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46295 (local address 127.22.121.1:35853)
0811 06:17:19.359523 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:19.359569 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:19.361334 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:19.362032 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:19.362042 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:19.362733 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:19.363517 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:19.363543 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.364559 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.364565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:19.366300 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:19.366315 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.367010 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.367017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:19.367283 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:19.367926 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:19.367953 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:19.370205 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:19.373560 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:19.373566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:19.373583 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:19.373975 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:19.374339 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:19.374343 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:19.374345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:19.374505 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:19.375063 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:19.375068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:19.375380 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 06:17:19.375602 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 06:17:19.376005 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":312,"threads_started":1}
I20260811 06:17:19.378683 31114 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46295
I20260811 06:17:19.378949 31114 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:19.379601 31114 heartbeater.cc:511] Master 127.22.121.62:46295 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:17:19.381534 30785 ts_manager.cc:194] Registered new tserver with Master: d3cd92a972e445ecb1cabf176685c81e (127.22.121.1:43295)
I20260811 06:17:19.383025 30785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:35853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:19.404220 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:19.712668 31126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:19.712903 31126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:19.713040 31126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:19.719831 31126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:19.719924 31126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:19.720098 31126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:19.729139 31126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:19.730288 31126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:19.731505 31126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:19.743501 31132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:19.743541 31134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:19.743584 31137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:19.744278 31126 server_base.cc:1034] running on GCE node
I20260811 06:17:19.744771 31126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:19.745553 31126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:19.746740 31126 hybrid_clock.cc:648] HybridClock initialized: now 1786429039746705 us; error 31 us; skew 500 ppm
I20260811 06:17:19.747026 31126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:19 dist-test-slave-gjrz krb5kdc[30696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429039, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:19.749935 31126 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:19.750952 31126 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:19.751042 31126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:19.751372 31126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:19.754017 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "739dce312beb460fb812536dcf32381d"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "129015449dae2e5dc27847097754f440"
server_key_iv: "08918ab8bf5869fcbaec7d78dd604135"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.754763 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "739dce312beb460fb812536dcf32381d"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "129015449dae2e5dc27847097754f440"
server_key_iv: "08918ab8bf5869fcbaec7d78dd604135"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.759342 31126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.007s
I20260811 06:17:19.762449 31150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.763743 31126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:17:19.763888 31126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "739dce312beb460fb812536dcf32381d"
format_stamp: "Formatted at 2026-08-11 06:17:19 on dist-test-slave-gjrz"
server_key: "129015449dae2e5dc27847097754f440"
server_key_iv: "08918ab8bf5869fcbaec7d78dd604135"
server_key_version: "encryptionkey@0"
I20260811 06:17:19.764073 31126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:19.783447 31126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:19.786839 31126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:19.787061 31126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:19.787982 31126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:19.789309 31126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:19.789402 31126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.789517 31126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:19.789592 31126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:19.820942 31126 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:36573
I20260811 06:17:19.821000 31328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:36573 every 8 connection(s)
I20260811 06:17:19.821708 31126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:19.828622 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31126
I20260811 06:17:19.828797 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:19.829288 23012 external_mini_cluster.cc:1428] Setting key 38ba3f6eb7840477e8526d235d7ede6a
Aug 11 06:17:19 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429039, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:19.840591 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:19.823979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:36079 (local address 127.22.121.62:46295)
0811 06:17:19.824141 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:19.824146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:19.825203 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 06:17:19.826916 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:19.826922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:19.827016 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:19.827102 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:19.828548 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.829474 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.831221 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.831675 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.831787 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:19.835241 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:19.835246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:19.835249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:19.835306 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:19.837691 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:19.838416 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:19.838422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:19.838425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:19.838591 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:19.838947 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:19.838951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:19.838954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:19.839303 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:19.839445 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:19.840040 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 06:17:19.840256 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:17:19.841239 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:19.824363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46295 (local address 127.22.121.2:36079)
0811 06:17:19.825039 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:19.825084 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:19.826655 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:19.827357 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:19.827365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:19.827857 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:19.828384 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:19.828400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.829629 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.829633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:19.831087 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:19.831099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:19.831881 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:19.831888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:19.832094 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:19.832673 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:19.832694 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:19.835021 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:19.837818 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:19.837827 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:19.837842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:19.838310 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:19.838688 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:19.838691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:19.838693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:19.838855 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:19.839418 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:19.839423 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:19.839701 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 06:17:19.839914 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 06:17:19.840272 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":275,"threads_started":1}
I20260811 06:17:19.842628 31331 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46295
I20260811 06:17:19.842885 31331 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:19.843515 31331 heartbeater.cc:511] Master 127.22.121.62:46295 requested a full tablet report, sending...
I20260811 06:17:19.844847 30785 ts_manager.cc:194] Registered new tserver with Master: 739dce312beb460fb812536dcf32381d (127.22.121.2:36573)
I20260811 06:17:19.845659 30785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:36079
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:19.872138 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:20.179061 31338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:20.179355 31338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:20.179513 31338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:20.186425 31338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:20.186517 31338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:20.186656 31338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:20.195873 31338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:20.197016 31338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:20.198226 31338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:20.209841 31351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:20.209952 31350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:20.209878 31353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:20.210872 31338 server_base.cc:1034] running on GCE node
I20260811 06:17:20.211450 31338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:20.212261 31338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:20.213464 31338 hybrid_clock.cc:648] HybridClock initialized: now 1786429040213428 us; error 23 us; skew 500 ppm
I20260811 06:17:20.213747 31338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:20 dist-test-slave-gjrz krb5kdc[30696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:20.220592 31338 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:20.221606 31338 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:20.221702 31338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:20.222041 31338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:20.224910 31338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "fc47c623ea214f22ac8e6036089e3bd2"
format_stamp: "Formatted at 2026-08-11 06:17:20 on dist-test-slave-gjrz"
server_key: "a3e73facaaa582359d0c37b084e93c8a"
server_key_iv: "aa9aad0b35a53ffa0ccc27182bbd4149"
server_key_version: "encryptionkey@0"
I20260811 06:17:20.225672 31338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "fc47c623ea214f22ac8e6036089e3bd2"
format_stamp: "Formatted at 2026-08-11 06:17:20 on dist-test-slave-gjrz"
server_key: "a3e73facaaa582359d0c37b084e93c8a"
server_key_iv: "aa9aad0b35a53ffa0ccc27182bbd4149"
server_key_version: "encryptionkey@0"
I20260811 06:17:20.230269 31338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 06:17:20.233407 31367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:20.234551 31338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:17:20.234709 31338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "fc47c623ea214f22ac8e6036089e3bd2"
format_stamp: "Formatted at 2026-08-11 06:17:20 on dist-test-slave-gjrz"
server_key: "a3e73facaaa582359d0c37b084e93c8a"
server_key_iv: "aa9aad0b35a53ffa0ccc27182bbd4149"
server_key_version: "encryptionkey@0"
I20260811 06:17:20.234894 31338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:20.247387 31338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:20.250941 31338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:20.251176 31338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:20.252029 31338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:20.253214 31338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:20.253306 31338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:20.253407 31338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:20.253479 31338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:20.284325 31338 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:45297
I20260811 06:17:20.284379 31560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:45297 every 8 connection(s)
I20260811 06:17:20.285068 31338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:20.286898 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31338
I20260811 06:17:20.287071 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:20.287607 23012 external_mini_cluster.cc:1428] Setting key 89cd1586808fa81fb7261d9aaec316a0
Aug 11 06:17:20 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:20.303453 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.287289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:54513 (local address 127.22.121.62:46295)
0811 06:17:20.287472 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.287478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.288625 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 06:17:20.290724 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.290728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.290843 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.290928 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:20.292357 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.293082 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.294562 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.294952 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.295055 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.298074 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:20.298079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:20.298083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:20.298113 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:20.300403 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.301205 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.301211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.301214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.301374 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.301837 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.301842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.301846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.302213 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:20.302363 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.302983 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.303153 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:17:20.304261 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.287736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46295 (local address 127.22.121.3:54513)
0811 06:17:20.288470 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.288527 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.290459 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.291160 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.291168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.291685 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:20.292206 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.292223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.293228 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.293233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.294432 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.294444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.295153 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.295162 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.295431 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.295945 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:20.295965 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:20.297877 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:20.300531 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:20.300540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:20.300554 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:20.301077 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:20.301490 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:20.301494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:20.301497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:20.301720 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:20.302344 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:20.302351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:20.302689 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.302889 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.303217 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260811 06:17:20.305595 31561 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46295
I20260811 06:17:20.305892 31561 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:20.306495 31561 heartbeater.cc:511] Master 127.22.121.62:46295 requested a full tablet report, sending...
I20260811 06:17:20.307794 30785 ts_manager.cc:194] Registered new tserver with Master: fc47c623ea214f22ac8e6036089e3bd2 (127.22.121.3:45297)
I20260811 06:17:20.308542 30785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:54513
I20260811 06:17:20.314209 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:17:20.329890 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.318418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.22.121.62:46295)
0811 06:17:20.318572 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.318576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.319054 (+ 478us) server_negotiation.cc:407] Connection header received
0811 06:17:20.319211 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.319215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.319314 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.319386 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:20.320745 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.321450 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.322692 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.323159 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.323281 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.325268 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:20.325273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:20.325276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:20.325305 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:20.327513 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.328095 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.328100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.328103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.328222 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.328553 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.328559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.328562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.328866 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:20.329107 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.329418 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.329628 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:17:20.333585 30785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:20.340323 30785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:20.365811 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.351718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:36573 (local address 127.0.0.1:60450)
0811 06:17:20.352480 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.352529 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.352706 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.353784 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.353793 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.353838 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:20.354243 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.354255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.361583 (+ 7328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.361588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.363515 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.363530 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.364857 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.364864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.365075 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.365126 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.365245 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.365417 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":325,"threads_started":1}
I20260811 06:17:20.366786 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.351979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.22.121.2:36573)
0811 06:17:20.353093 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.353102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.353130 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:17:20.353228 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.353236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.353480 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.353747 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:20.359880 (+ 6133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.361418 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.363716 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.364606 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.364782 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.364970 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.366323 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.366489 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":951,"thread_start_us":1216,"threads_started":1}
I20260811 06:17:20.367391 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.352908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.22.121.3:45297)
0811 06:17:20.358337 (+ 5429us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.358346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.359047 (+ 701us) server_negotiation.cc:407] Connection header received
0811 06:17:20.359145 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.359153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.359426 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.359628 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:20.360542 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.362140 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.364715 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.366288 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.366405 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.366619 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.366913 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.367076 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5256,"thread_start_us":5074,"threads_started":1}
I20260811 06:17:20.367472 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.352288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:45297 (local address 127.0.0.1:53848)
0811 06:17:20.358784 (+ 6496us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.358801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.358922 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.359895 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.359900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.359925 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:20.360328 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.360340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.362303 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.362308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.364533 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.364548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.366564 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.366571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.366729 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.366762 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.366854 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.367082 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6371,"mutex_wait_us":27,"thread_start_us":7360,"threads_started":1}
I20260811 06:17:20.370414 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.352440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43295 (local address 127.0.0.1:40224)
0811 06:17:20.359905 (+ 7465us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.359921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.360129 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.361033 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.361038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.361063 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:20.361368 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.361379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.363862 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.363867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.365925 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.365943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.368719 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.368726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.368891 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.368923 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.368999 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.370063 (+ 1064us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7332,"mutex_wait_us":2,"thread_start_us":6068,"threads_started":1}
I20260811 06:17:20.370415 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.353039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.22.121.1:43295)
0811 06:17:20.353568 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.353577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.360029 (+ 6452us) server_negotiation.cc:407] Connection header received
0811 06:17:20.360330 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.360337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.360547 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.360730 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:20.362088 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.363654 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.366109 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.368502 (+ 2393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.368630 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.369890 (+ 1260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.369976 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.370121 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":363,"threads_started":1}
I20260811 06:17:20.370777 31223 tablet_service.cc:1467] Processing CreateTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 (DEFAULT_TABLE table=test-table [id=38208f55bcfb439dae3ab356d13f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:20.370956 31447 tablet_service.cc:1467] Processing CreateTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 (DEFAULT_TABLE table=test-table [id=38208f55bcfb439dae3ab356d13f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:20.372085 31223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4348e1b03ba34eb982b028cab1cfe3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:20.372963 31447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4348e1b03ba34eb982b028cab1cfe3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:20.373377 31005 tablet_service.cc:1467] Processing CreateTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 (DEFAULT_TABLE table=test-table [id=38208f55bcfb439dae3ab356d13f467f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:20.374414 31005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4348e1b03ba34eb982b028cab1cfe3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:20.384488 31599 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Bootstrap starting.
I20260811 06:17:20.385802 31114 heartbeater.cc:503] Master 127.22.121.62:46295 was elected leader, sending a full tablet report...
I20260811 06:17:20.388249 31599 tablet_bootstrap.cc:654] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:20.389453 31599 log.cc:824] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:20.389523 31602 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Bootstrap starting.
I20260811 06:17:20.394390 31602 tablet_bootstrap.cc:654] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:20.395473 31601 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Bootstrap starting.
I20260811 06:17:20.395645 31602 log.cc:824] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:20.397116 31599 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: No bootstrap required, opened a new log
I20260811 06:17:20.397397 31599 ts_tablet_manager.cc:1397] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260811 06:17:20.398792 31601 tablet_bootstrap.cc:654] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:20.400009 31601 log.cc:824] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:20.402842 31601 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: No bootstrap required, opened a new log
I20260811 06:17:20.403119 31601 ts_tablet_manager.cc:1397] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:17:20.403193 31602 tablet_bootstrap.cc:492] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: No bootstrap required, opened a new log
I20260811 06:17:20.403503 31602 ts_tablet_manager.cc:1397] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 06:17:20.405682 31599 raft_consensus.cc:348] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.406093 31599 raft_consensus.cc:374] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:20.406235 31599 raft_consensus.cc:729] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd92a972e445ecb1cabf176685c81e, State: Initialized, Role: FOLLOWER
I20260811 06:17:20.406764 31599 consensus_queue.cc:260] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.408766 31599 ts_tablet_manager.cc:1428] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 06:17:20.413477 31601 raft_consensus.cc:348] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.413882 31601 raft_consensus.cc:374] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:20.414000 31601 raft_consensus.cc:729] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739dce312beb460fb812536dcf32381d, State: Initialized, Role: FOLLOWER
I20260811 06:17:20.413968 31602 raft_consensus.cc:348] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.414343 31602 raft_consensus.cc:374] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:20.414439 31602 raft_consensus.cc:729] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc47c623ea214f22ac8e6036089e3bd2, State: Initialized, Role: FOLLOWER
I20260811 06:17:20.414579 31601 consensus_queue.cc:260] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.415020 31602 consensus_queue.cc:260] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.415642 31331 heartbeater.cc:503] Master 127.22.121.62:46295 was elected leader, sending a full tablet report...
I20260811 06:17:20.416320 31601 ts_tablet_manager.cc:1428] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260811 06:17:20.417230 31561 heartbeater.cc:503] Master 127.22.121.62:46295 was elected leader, sending a full tablet report...
I20260811 06:17:20.417924 31602 ts_tablet_manager.cc:1428] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260811 06:17:20.536367 31562 tablet.cc:2369] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:17:20.573397 31332 tablet.cc:2369] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:20.605764 31612 raft_consensus.cc:482] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:20.606022 31612 raft_consensus.cc:504] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.607671 31612 leader_election.cc:288] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 739dce312beb460fb812536dcf32381d (127.22.121.2:36573), d3cd92a972e445ecb1cabf176685c81e (127.22.121.1:43295)
W20260811 06:17:20.608132 31115 tablet.cc:2369] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:20.613212 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.608052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:43295 (local address 127.22.121.3:47097)
0811 06:17:20.608351 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.608370 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.608524 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.608866 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.608871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.608898 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:20.609100 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.609119 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.610161 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.610166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.611491 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.611503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.612539 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.612545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.612720 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.612745 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.612823 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.612948 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260811 06:17:20.613216 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.608203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:47097 (local address 127.22.121.1:43295)
0811 06:17:20.608457 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.608463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.608481 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:20.608669 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.608673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.608732 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.608806 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:20.609262 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.610005 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.611643 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.612385 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.612533 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.612603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.612861 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.612991 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 06:17:20.614141 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.608613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:36573 (local address 127.22.121.3:52789)
0811 06:17:20.609373 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:20.609387 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:20.609505 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:20.609792 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:20.609796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:20.609818 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:20.610122 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.610132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.611078 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.611082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.612387 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:20.612398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.613529 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.613536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:20.613700 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.613732 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:17:20.613830 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:17:20.613912 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":457,"threads_started":1}
I20260811 06:17:20.614238 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.608086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:52789 (local address 127.22.121.2:36573)
0811 06:17:20.608281 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.608287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.609475 (+ 1188us) server_negotiation.cc:407] Connection header received
0811 06:17:20.609634 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.609638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.609695 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.609768 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:20.610266 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.610937 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.612532 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.613275 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.613416 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.613476 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.613870 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.613998 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:17:20.614280 31036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4348e1b03ba34eb982b028cab1cfe3c0" candidate_uuid: "fc47c623ea214f22ac8e6036089e3bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd92a972e445ecb1cabf176685c81e" is_pre_election: true
I20260811 06:17:20.614765 31036 raft_consensus.cc:2454] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc47c623ea214f22ac8e6036089e3bd2 in term 0.
I20260811 06:17:20.615084 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4348e1b03ba34eb982b028cab1cfe3c0" candidate_uuid: "fc47c623ea214f22ac8e6036089e3bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739dce312beb460fb812536dcf32381d" is_pre_election: true
I20260811 06:17:20.615422 31371 leader_election.cc:302] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3cd92a972e445ecb1cabf176685c81e, fc47c623ea214f22ac8e6036089e3bd2; no voters:
I20260811 06:17:20.615615 31259 raft_consensus.cc:2454] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc47c623ea214f22ac8e6036089e3bd2 in term 0.
I20260811 06:17:20.615768 31612 raft_consensus.cc:2781] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:20.615918 31612 raft_consensus.cc:482] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:20.616052 31612 raft_consensus.cc:3037] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:20.618213 31612 raft_consensus.cc:504] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.618729 31612 leader_election.cc:288] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 739dce312beb460fb812536dcf32381d (127.22.121.2:36573), d3cd92a972e445ecb1cabf176685c81e (127.22.121.1:43295)
I20260811 06:17:20.619103 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4348e1b03ba34eb982b028cab1cfe3c0" candidate_uuid: "fc47c623ea214f22ac8e6036089e3bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "739dce312beb460fb812536dcf32381d"
I20260811 06:17:20.619204 31036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4348e1b03ba34eb982b028cab1cfe3c0" candidate_uuid: "fc47c623ea214f22ac8e6036089e3bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd92a972e445ecb1cabf176685c81e"
I20260811 06:17:20.619324 31259 raft_consensus.cc:3037] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:20.619415 31036 raft_consensus.cc:3037] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:20.622373 31259 raft_consensus.cc:2454] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc47c623ea214f22ac8e6036089e3bd2 in term 1.
I20260811 06:17:20.622373 31036 raft_consensus.cc:2454] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc47c623ea214f22ac8e6036089e3bd2 in term 1.
I20260811 06:17:20.622725 31371 leader_election.cc:302] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3cd92a972e445ecb1cabf176685c81e, fc47c623ea214f22ac8e6036089e3bd2; no voters:
I20260811 06:17:20.622977 31612 raft_consensus.cc:2781] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:20.623399 31612 raft_consensus.cc:686] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 1 LEADER]: Becoming Leader. State: Replica: fc47c623ea214f22ac8e6036089e3bd2, State: Running, Role: LEADER
I20260811 06:17:20.623898 31612 consensus_queue.cc:237] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } }
I20260811 06:17:20.627951 30785 catalog_manager.cc:5495] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc47c623ea214f22ac8e6036089e3bd2 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "fc47c623ea214f22ac8e6036089e3bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc47c623ea214f22ac8e6036089e3bd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45297 } health_report { overall_health: HEALTHY } } }
I20260811 06:17:20.659018 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.654514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.22.121.3:45297)
0811 06:17:20.654662 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.654666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.654841 (+ 175us) server_negotiation.cc:407] Connection header received
0811 06:17:20.654997 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.655000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.655067 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.655185 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:20.655487 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.656179 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.657396 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.657760 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.657843 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.658215 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:20.658642 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:20.658739 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.658791 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.658846 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 06:17:20.666571 31036 raft_consensus.cc:1261] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 1 FOLLOWER]: Refusing update from remote peer fc47c623ea214f22ac8e6036089e3bd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:20.666569 31259 raft_consensus.cc:1261] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 1 FOLLOWER]: Refusing update from remote peer fc47c623ea214f22ac8e6036089e3bd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:20.667583 31617 consensus_queue.cc:1035] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd92a972e445ecb1cabf176685c81e" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 43295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:20.667917 31612 consensus_queue.cc:1035] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "739dce312beb460fb812536dcf32381d" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:20.678682 31623 mvcc.cc:205] Tried to move back new op lower bound from 7317213350560620544 to 7317213350398992384. Current Snapshot: MvccSnapshot[applied={T|T < 7317213350560620544}]
I20260811 06:17:20.683449 31622 mvcc.cc:205] Tried to move back new op lower bound from 7317213350560620544 to 7317213350398992384. Current Snapshot: MvccSnapshot[applied={T|T < 7317213350560620544}]
I20260811 06:17:20.695453 30785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:20.695667 30785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:20.698715 30785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fd3db180c6b42d699eee7289e29d8d4: Sending DeleteTablet for 3 replicas of tablet 4348e1b03ba34eb982b028cab1cfe3c0
I20260811 06:17:20.699677 31223 tablet_service.cc:1514] Processing DeleteTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:20 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:60450
I20260811 06:17:20.699750 31005 tablet_service.cc:1514] Processing DeleteTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:20 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:40224
I20260811 06:17:20.700002 31447 tablet_service.cc:1514] Processing DeleteTablet for tablet 4348e1b03ba34eb982b028cab1cfe3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:20 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:53848
I20260811 06:17:20.702224 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30895
I20260811 06:17:20.715387 31633 tablet_replica.cc:330] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: stopping tablet replica
I20260811 06:17:20.715773 31633 raft_consensus.cc:2229] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:20.716251 31633 raft_consensus.cc:2258] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:20.718231 31633 ts_tablet_manager.cc:1905] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:20.725446 31633 ts_tablet_manager.cc:1918] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:20.725610 31633 log.cc:1196] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/4348e1b03ba34eb982b028cab1cfe3c0
I20260811 06:17:20.726150 31633 ts_tablet_manager.cc:1939] T 4348e1b03ba34eb982b028cab1cfe3c0 P 739dce312beb460fb812536dcf32381d: Deleting consensus metadata
I20260811 06:17:20.727877 30764 catalog_manager.cc:4841] TS 739dce312beb460fb812536dcf32381d (127.22.121.2:36573): tablet 4348e1b03ba34eb982b028cab1cfe3c0 (table test-table [id=38208f55bcfb439dae3ab356d13f467f]) successfully deleted
I20260811 06:17:20.730046 31634 tablet_replica.cc:330] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: stopping tablet replica
I20260811 06:17:20.730515 31634 raft_consensus.cc:2229] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:20.731124 31634 raft_consensus.cc:2258] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:20.731734 31635 tablet_replica.cc:330] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: stopping tablet replica
I20260811 06:17:20.732113 31635 raft_consensus.cc:2229] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:20.732662 31635 raft_consensus.cc:2258] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:20.734200 31634 ts_tablet_manager.cc:1905] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:20.734371 31635 ts_tablet_manager.cc:1905] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:17:20 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:20.741096 31634 ts_tablet_manager.cc:1918] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:20.741110 31635 ts_tablet_manager.cc:1918] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:20.741248 31635 log.cc:1196] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/4348e1b03ba34eb982b028cab1cfe3c0
I20260811 06:17:20.741403 31634 log.cc:1196] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/4348e1b03ba34eb982b028cab1cfe3c0
I20260811 06:17:20.741796 31635 ts_tablet_manager.cc:1939] T 4348e1b03ba34eb982b028cab1cfe3c0 P fc47c623ea214f22ac8e6036089e3bd2: Deleting consensus metadata
I20260811 06:17:20.741986 31634 ts_tablet_manager.cc:1939] T 4348e1b03ba34eb982b028cab1cfe3c0 P d3cd92a972e445ecb1cabf176685c81e: Deleting consensus metadata
I20260811 06:17:20.743517 30766 catalog_manager.cc:4841] TS fc47c623ea214f22ac8e6036089e3bd2 (127.22.121.3:45297): tablet 4348e1b03ba34eb982b028cab1cfe3c0 (table test-table [id=38208f55bcfb439dae3ab356d13f467f]) successfully deleted
I20260811 06:17:20.743537 30765 catalog_manager.cc:4841] TS d3cd92a972e445ecb1cabf176685c81e (127.22.121.1:43295): tablet 4348e1b03ba34eb982b028cab1cfe3c0 (table test-table [id=38208f55bcfb439dae3ab356d13f467f]) successfully deleted
I20260811 06:17:20.745898 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.703493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.22.121.1:43295)
0811 06:17:20.703669 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.703676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.727184 (+ 23508us) server_negotiation.cc:407] Connection header received
0811 06:17:20.727608 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.727614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.727693 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.727777 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:20.729603 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.730984 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.732906 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.733539 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.733636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.739005 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:20.739018 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:20.739034 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:20.739098 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:20.742946 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.743955 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.743962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.743964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.744093 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.744466 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.744469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.744471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.744811 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:20.745028 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.745430 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.745587 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:17:20.746541 31106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:20.857635 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30895
I20260811 06:17:20.878585 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31126
Aug 11 06:17:20 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:20.894095 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:20.879419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.22.121.2:36573)
0811 06:17:20.879619 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:20.879625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:20.879674 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:17:20.879742 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:20.879748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:20.879828 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:20.879928 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:20.881057 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.882192 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.883470 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:20.884103 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:20.884237 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:20.888129 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:20.888145 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:20.888164 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:20.888208 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:20.891614 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.892207 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.892216 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.892220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.892367 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:20.892664 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:20.892669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:20.892673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:20.893118 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:20.893382 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:20.893543 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:17:20.893744 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:17:20.894793 31322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:20.982340 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31126
I20260811 06:17:21.008311 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31338
Aug 11 06:17:21 dist-test-slave-gjrz krb5kdc[30696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:21.026067 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:21.009147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.22.121.3:45297)
0811 06:17:21.009358 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:21.009364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:21.009388 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:17:21.009516 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:21.009521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:21.009637 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:21.009756 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:21.010969 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:21.012260 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:21.013535 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:21.014226 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:21.014359 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:21.018792 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:21.018813 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:21.018835 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:21.018882 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:21.022729 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:21.023601 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:21.023611 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:21.023615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:21.023841 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:21.024413 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:21.024418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:21.024457 (+ 39us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:21.025001 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:21.025262 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:21.025595 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:17:21.025752 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 06:17:21.027015 31555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:21.148342 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31338
I20260811 06:17:21.176091 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30735
I20260811 06:17:21.177222 30841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:21.280983 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30735
2026-08-11T06:17:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4299 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T06:17:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:21Z Disabled control of system clock
I20260811 06:17:21.353488 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42017
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:17:21.659570 31654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:21.659824 31654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:21.659957 31654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:21.666797 31654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:21.666888 31654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:21.666958 31654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:21.667037 31654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:21.667094 31654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:21.675674 31654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:42017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:21.676726 31654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:21.678331 31654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:21.689271 31666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:21.689338 31661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:21.689302 31663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:21.690140 31654 server_base.cc:1034] running on GCE node
I20260811 06:17:21.690840 31654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:21.692171 31654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:21.693383 31654 hybrid_clock.cc:648] HybridClock initialized: now 1786429041693339 us; error 32 us; skew 500 ppm
I20260811 06:17:21.693708 31654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:21.695801 31654 webserver.cc:460] Webserver started at http://127.22.121.62:35473/ using document root <none> and password file <none>
I20260811 06:17:21.696430 31654 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:21.696520 31654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:21.696817 31654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:21.699522 31654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "280592b376de4e42b908a5f8702e1137"
format_stamp: "Formatted at 2026-08-11 06:17:21 on dist-test-slave-gjrz"
server_key: "979695ba9165006ac6276666d0954b42"
server_key_iv: "293999fc0cd577c9d4ccf5e747fb0776"
server_key_version: "encryptionkey@0"
I20260811 06:17:21.700265 31654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "280592b376de4e42b908a5f8702e1137"
format_stamp: "Formatted at 2026-08-11 06:17:21 on dist-test-slave-gjrz"
server_key: "979695ba9165006ac6276666d0954b42"
server_key_iv: "293999fc0cd577c9d4ccf5e747fb0776"
server_key_version: "encryptionkey@0"
I20260811 06:17:21.705207 31654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:17:21.708321 31681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:21.709479 31654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260811 06:17:21.709628 31654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "280592b376de4e42b908a5f8702e1137"
format_stamp: "Formatted at 2026-08-11 06:17:21 on dist-test-slave-gjrz"
server_key: "979695ba9165006ac6276666d0954b42"
server_key_iv: "293999fc0cd577c9d4ccf5e747fb0776"
server_key_version: "encryptionkey@0"
I20260811 06:17:21.709817 31654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:21.715113 31654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:21.715801 31654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:21.716068 31654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:21.732370 31654 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:42017
I20260811 06:17:21.732424 31777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:42017 every 8 connection(s)
I20260811 06:17:21.733515 31654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:21.736903 31779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:21.737208 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31654
I20260811 06:17:21.737386 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:21.738020 23012 external_mini_cluster.cc:1428] Setting key bdbcbf90bb4f2a40ec0d4c4cfabf6168
I20260811 06:17:21.745710 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:21.739896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.22.121.62:42017)
0811 06:17:21.740884 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:21.740893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:21.740936 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:17:21.742479 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:21.742500 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:21.742882 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:21.743223 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:21.744226 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:21.744232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:21.744245 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:21.744259 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:21.744318 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:21.744337 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:21.744547 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:21.744674 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 06:17:21.744893 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":361,"threads_started":1}
I20260811 06:17:21.748483 31779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: Bootstrap starting.
I20260811 06:17:21.751363 31779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:21.752259 31779 log.cc:824] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:21.754485 31779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: No bootstrap required, opened a new log
I20260811 06:17:21.761269 31779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } }
I20260811 06:17:21.761579 31779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:21.761678 31779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280592b376de4e42b908a5f8702e1137, State: Initialized, Role: FOLLOWER
I20260811 06:17:21.762172 31779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } }
I20260811 06:17:21.762368 31779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:21.762506 31779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:21.762674 31779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:21.764799 31779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } }
I20260811 06:17:21.765206 31779 leader_election.cc:302] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 280592b376de4e42b908a5f8702e1137; no voters:
I20260811 06:17:21.765693 31779 leader_election.cc:288] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:21.765805 31789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:21.766430 31789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [term 1 LEADER]: Becoming Leader. State: Replica: 280592b376de4e42b908a5f8702e1137, State: Running, Role: LEADER
I20260811 06:17:21.766942 31789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } }
I20260811 06:17:21.767524 31779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:21.769443 31792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 280592b376de4e42b908a5f8702e1137. Latest consensus state: current_term: 1 leader_uuid: "280592b376de4e42b908a5f8702e1137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } } }
I20260811 06:17:21.769832 31792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:21.769760 31790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "280592b376de4e42b908a5f8702e1137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "280592b376de4e42b908a5f8702e1137" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42017 } } }
I20260811 06:17:21.770054 31790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:21.771566 31806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:21.776823 31806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:21.784996 31806 catalog_manager.cc:1347] Generated new cluster ID: 738b8f7ff7354e3682b0c4a1b56b5c47
I20260811 06:17:21.785079 31806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:21.797101 31806 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:21.797521 31806 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:21.805959 31806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: Generated new TSK 0
I20260811 06:17:21.806779 31806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:17:21.813978 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--builtin_ntp_servers=127.22.121.20:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:17:22.122951 31822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:22.123212 31822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:22.123358 31822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:22.130308 31822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:22.130419 31822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:22.130563 31822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:22.139231 31822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:22.140431 31822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:22.142167 31822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:22.154400 31835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.154449 31832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.154486 31830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:22.155256 31822 server_base.cc:1034] running on GCE node
I20260811 06:17:22.155822 31822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:22.156639 31822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:22.157831 31822 hybrid_clock.cc:648] HybridClock initialized: now 1786429042157780 us; error 44 us; skew 500 ppm
I20260811 06:17:22.158154 31822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:22.160239 31822 webserver.cc:460] Webserver started at http://127.22.121.1:37553/ using document root <none> and password file <none>
I20260811 06:17:22.160851 31822 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:22.160954 31822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:22.161258 31822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:22.163878 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "869020dcaa4346cd86e3e1d6786e46bc"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "160ef1fdd73b9ef65dff99a6fbcfe926"
server_key_iv: "7f89c4d00d6803fe6056f6f19f029153"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.164603 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "869020dcaa4346cd86e3e1d6786e46bc"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "160ef1fdd73b9ef65dff99a6fbcfe926"
server_key_iv: "7f89c4d00d6803fe6056f6f19f029153"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.169386 31822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:17:22.172684 31845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.173861 31822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:22.174028 31822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "869020dcaa4346cd86e3e1d6786e46bc"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "160ef1fdd73b9ef65dff99a6fbcfe926"
server_key_iv: "7f89c4d00d6803fe6056f6f19f029153"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.174220 31822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:22.179419 31822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:22.180161 31822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:22.180397 31822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:22.181232 31822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:22.182580 31822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:22.182674 31822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.182775 31822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:22.182842 31822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.210009 31822 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:46359
I20260811 06:17:22.210054 32044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:46359 every 8 connection(s)
I20260811 06:17:22.211166 31822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:22.212225 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31822
I20260811 06:17:22.212412 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:22.213122 23012 external_mini_cluster.cc:1428] Setting key 3c24dbd7fd11b4dc77d5b38cd1e5c30c
I20260811 06:17:22.216022 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--builtin_ntp_servers=127.22.121.20:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:17:22.218940 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:22.213641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:35511 (local address 127.22.121.62:42017)
0811 06:17:22.213824 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:22.213830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:22.215292 (+ 1462us) server_negotiation.cc:407] Connection header received
0811 06:17:22.216967 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:22.216972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:22.217032 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:22.217136 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:22.218282 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:22.218288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:22.218292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:22.218310 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:22.218347 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:22.218362 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:22.218472 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:22.218633 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:17:22.218691 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:17:22.219790 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:22.214102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:42017 (local address 127.22.121.1:35511)
0811 06:17:22.215133 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:22.215180 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:22.216717 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:22.217297 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:22.217304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:22.217387 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:22.218003 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:22.218006 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:22.218042 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:22.218122 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:22.218445 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:22.218451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:22.218456 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:17:22.218659 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 06:17:22.218838 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":309,"threads_started":1}
I20260811 06:17:22.221127 32048 heartbeater.cc:347] Connected to a master server at 127.22.121.62:42017
I20260811 06:17:22.221405 32048 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:22.222023 32048 heartbeater.cc:511] Master 127.22.121.62:42017 requested a full tablet report, sending...
I20260811 06:17:22.224000 31708 ts_manager.cc:194] Registered new tserver with Master: 869020dcaa4346cd86e3e1d6786e46bc (127.22.121.1:46359)
W20260811 06:17:22.526428 32052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:22.526690 32052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:22.526823 32052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:22.533775 32052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:22.533874 32052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:22.534024 32052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:22.542593 32052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:22.543707 32052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:22.545315 32052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:22.557376 32061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.557600 32059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.557421 32064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:22.558285 32052 server_base.cc:1034] running on GCE node
I20260811 06:17:22.558868 32052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:22.559753 32052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:22.560943 32052 hybrid_clock.cc:648] HybridClock initialized: now 1786429042560899 us; error 32 us; skew 500 ppm
I20260811 06:17:22.561218 32052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:22.563211 32052 webserver.cc:460] Webserver started at http://127.22.121.2:39753/ using document root <none> and password file <none>
I20260811 06:17:22.563848 32052 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:22.563938 32052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:22.564242 32052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:22.566860 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "e14a0a70d4ff41edb74cc1de4bac21de"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "a8565d053dfc783c293dfd7b5effbea6"
server_key_iv: "b67aa29b038a8b6bc8e54ee6576d4d46"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.567694 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "e14a0a70d4ff41edb74cc1de4bac21de"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "a8565d053dfc783c293dfd7b5effbea6"
server_key_iv: "b67aa29b038a8b6bc8e54ee6576d4d46"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.572517 32052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:17:22.575646 32078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.576802 32052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260811 06:17:22.576930 32052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e14a0a70d4ff41edb74cc1de4bac21de"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "a8565d053dfc783c293dfd7b5effbea6"
server_key_iv: "b67aa29b038a8b6bc8e54ee6576d4d46"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.577112 32052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:22.582417 32052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:22.583134 32052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:22.583397 32052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:22.584233 32052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:22.585487 32052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:22.585570 32052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.585646 32052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:22.585698 32052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.614276 32052 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:45051
I20260811 06:17:22.614358 32279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:45051 every 8 connection(s)
I20260811 06:17:22.615525 32052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:22.622958 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:22.617799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:50423 (local address 127.22.121.62:42017)
0811 06:17:22.617972 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:22.617976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:22.619307 (+ 1331us) server_negotiation.cc:407] Connection header received
0811 06:17:22.620918 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:22.620923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:22.620988 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:22.621078 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:22.622250 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:22.622256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:22.622261 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:22.622279 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:22.622311 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:22.622322 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:22.622423 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:22.622607 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 06:17:22.622676 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:22.623817 32286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:22.618290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:42017 (local address 127.22.121.2:50423)
0811 06:17:22.619150 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:22.619185 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:22.620661 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:22.621311 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:22.621319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:22.621404 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:22.621958 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:22.621960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:22.621995 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:22.622086 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:22.622408 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:22.622419 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:22.622426 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:17:22.622672 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 06:17:22.622902 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260811 06:17:22.624992 32282 heartbeater.cc:347] Connected to a master server at 127.22.121.62:42017
I20260811 06:17:22.625252 32282 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:22.625331 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32052
I20260811 06:17:22.625514 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:22.625814 32282 heartbeater.cc:511] Master 127.22.121.62:42017 requested a full tablet report, sending...
I20260811 06:17:22.625947 23012 external_mini_cluster.cc:1428] Setting key 827c772f17d652160317d75174d5948c
I20260811 06:17:22.627135 31706 ts_manager.cc:194] Registered new tserver with Master: e14a0a70d4ff41edb74cc1de4bac21de (127.22.121.2:45051)
I20260811 06:17:22.628522 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--builtin_ntp_servers=127.22.121.20:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:17:22.932806 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:22.933044 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:22.933151 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:22.940021 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:22.940105 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:22.940231 32287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:22.948751 32287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:42017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:22.949836 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:22.951478 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:22.963889 32301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.963889 32296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:22.963922 32295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:22.964775 32287 server_base.cc:1034] running on GCE node
I20260811 06:17:22.965305 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:22.966164 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:22.967341 32287 hybrid_clock.cc:648] HybridClock initialized: now 1786429042967291 us; error 42 us; skew 500 ppm
I20260811 06:17:22.967639 32287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:22.969755 32287 webserver.cc:460] Webserver started at http://127.22.121.3:33363/ using document root <none> and password file <none>
I20260811 06:17:22.970495 32287 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:22.970597 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:22.971015 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:22.973774 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "627366f870aa4d18b59947a2a59937b1"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "be32fb56d11668ae3a16b6ec54e02f60"
server_key_iv: "ddf65c10411d7c722dbe674b8ff2fd06"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.974570 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "627366f870aa4d18b59947a2a59937b1"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "be32fb56d11668ae3a16b6ec54e02f60"
server_key_iv: "ddf65c10411d7c722dbe674b8ff2fd06"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.979669 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:17:22.984009 32310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.985530 32287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:22.985695 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "627366f870aa4d18b59947a2a59937b1"
format_stamp: "Formatted at 2026-08-11 06:17:22 on dist-test-slave-gjrz"
server_key: "be32fb56d11668ae3a16b6ec54e02f60"
server_key_iv: "ddf65c10411d7c722dbe674b8ff2fd06"
server_key_version: "encryptionkey@0"
I20260811 06:17:22.985921 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:22.993218 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:22.994127 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:22.994431 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:22.995539 32287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:22.997429 32287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:22.997538 32287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:22.997648 32287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:22.997720 32287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:23.035429 32287 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:39051
I20260811 06:17:23.035482 32495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:39051 every 8 connection(s)
I20260811 06:17:23.036936 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:23.046250 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.040873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:40131 (local address 127.22.121.62:42017)
0811 06:17:23.041102 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.041108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.041714 (+ 606us) server_negotiation.cc:407] Connection header received
0811 06:17:23.044021 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.044028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.044093 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.044188 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.045515 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.045522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.045544 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.045565 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.045610 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.045627 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.045734 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.045927 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.045992 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:17:23.046856 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32287
I20260811 06:17:23.047049 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:23.047390 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.040364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:42017 (local address 127.22.121.3:40131)
0811 06:17:23.041565 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.041618 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.043733 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.044425 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.044436 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.044546 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.045245 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.045248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.045282 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.045361 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.045713 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.045724 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.045732 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.045976 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.046213 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":456,"threads_started":1}
I20260811 06:17:23.047627 23012 external_mini_cluster.cc:1428] Setting key 9418d17cfb3c4284103c9cc67eca054a
I20260811 06:17:23.051656 32502 heartbeater.cc:347] Connected to a master server at 127.22.121.62:42017
I20260811 06:17:23.051904 32502 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:23.052418 32502 heartbeater.cc:511] Master 127.22.121.62:42017 requested a full tablet report, sending...
I20260811 06:17:23.053675 31706 ts_manager.cc:194] Registered new tserver with Master: 627366f870aa4d18b59947a2a59937b1 (127.22.121.3:39051)
I20260811 06:17:23.062207 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:17:23.068552 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.066310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.22.121.62:42017)
0811 06:17:23.066448 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.066454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.066902 (+ 448us) server_negotiation.cc:407] Connection header received
0811 06:17:23.067068 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.067072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.067120 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.067201 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.068009 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.068015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.068020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.068039 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.068073 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.068085 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.068181 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.068257 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.068300 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:17:23.072232 31706 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:23.078902 31706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:23.105641 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.090689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.22.121.3:39051)
0811 06:17:23.097548 (+ 6859us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.097558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.102241 (+ 4683us) server_negotiation.cc:407] Connection header received
0811 06:17:23.102322 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.102330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.102549 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.103292 (+ 743us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.104717 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.104725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.104739 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.104755 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.104809 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.104831 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.105036 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.105161 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.105254 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":306,"threads_started":1}
I20260811 06:17:23.105978 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.091668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.22.121.1:46359)
0811 06:17:23.102283 (+ 10615us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.102291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.102876 (+ 585us) server_negotiation.cc:407] Connection header received
0811 06:17:23.102950 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.102957 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.103165 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.103317 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.105132 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.105141 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.105153 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.105173 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.105232 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.105256 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.105371 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.105581 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.105695 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10479,"thread_start_us":10356,"threads_started":1}
I20260811 06:17:23.105973 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.089440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:45051 (local address 127.0.0.1:60042)
0811 06:17:23.090348 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.090382 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.090545 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.103346 (+ 12801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.103351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.103374 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.104081 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.104084 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.104100 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.104143 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.105428 (+ 1285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.105433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.105438 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.105562 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.105616 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":1212,"threads_started":1}
I20260811 06:17:23.106101 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.090146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:39051 (local address 127.0.0.1:36342)
0811 06:17:23.102030 (+ 11884us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.102050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.102164 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.102766 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.102773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.102863 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.103648 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.103652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.103681 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.103762 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.104981 (+ 1219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.104994 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.105002 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.105180 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.105263 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11728,"thread_start_us":11580,"threads_started":1}
I20260811 06:17:23.106729 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.091410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:46359 (local address 127.0.0.1:54050)
0811 06:17:23.102667 (+ 11257us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.102684 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.102769 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.104256 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.104261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.104283 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.104918 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.104922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.104937 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.104978 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.105426 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.105430 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.105435 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.106351 (+ 916us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.106402 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11128,"thread_start_us":10997,"threads_started":1}
I20260811 06:17:23.106942 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.089678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.22.121.2:45051)
0811 06:17:23.102918 (+ 13240us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.102924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.102947 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:17:23.103000 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.103005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.103152 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.103739 (+ 587us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.105184 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.105193 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.105205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.105230 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.105282 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.105302 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.106457 (+ 1155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.106532 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.106616 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13095,"thread_start_us":12981,"threads_started":1}
I20260811 06:17:23.109802 32389 tablet_service.cc:1467] Processing CreateTablet for tablet f3d7e9907707464b9de892d755a2e986 (DEFAULT_TABLE table=test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:23.110028 31938 tablet_service.cc:1467] Processing CreateTablet for tablet f3d7e9907707464b9de892d755a2e986 (DEFAULT_TABLE table=test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:23.110301 32160 tablet_service.cc:1467] Processing CreateTablet for tablet f3d7e9907707464b9de892d755a2e986 (DEFAULT_TABLE table=test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:23.111193 32389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d7e9907707464b9de892d755a2e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:23.111349 31938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d7e9907707464b9de892d755a2e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:23.111519 32160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d7e9907707464b9de892d755a2e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:23.122915 32537 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Bootstrap starting.
I20260811 06:17:23.125368 32539 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Bootstrap starting.
I20260811 06:17:23.126219 32537 tablet_bootstrap.cc:654] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:23.127460 32537 log.cc:824] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:23.127986 32538 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Bootstrap starting.
I20260811 06:17:23.128670 32539 tablet_bootstrap.cc:654] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:23.129582 32539 log.cc:824] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:23.131251 32538 tablet_bootstrap.cc:654] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:23.131392 32537 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: No bootstrap required, opened a new log
I20260811 06:17:23.131665 32537 ts_tablet_manager.cc:1397] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 06:17:23.132404 32538 log.cc:824] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:23.133759 32539 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: No bootstrap required, opened a new log
I20260811 06:17:23.133997 32539 ts_tablet_manager.cc:1397] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:17:23.134675 32538 tablet_bootstrap.cc:492] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: No bootstrap required, opened a new log
I20260811 06:17:23.134882 32538 ts_tablet_manager.cc:1397] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 06:17:23.141337 32537 raft_consensus.cc:348] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.141750 32537 raft_consensus.cc:374] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:23.141849 32537 raft_consensus.cc:729] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627366f870aa4d18b59947a2a59937b1, State: Initialized, Role: FOLLOWER
I20260811 06:17:23.141954 32538 raft_consensus.cc:348] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.142340 32538 raft_consensus.cc:374] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:23.142444 32538 raft_consensus.cc:729] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869020dcaa4346cd86e3e1d6786e46bc, State: Initialized, Role: FOLLOWER
I20260811 06:17:23.142441 32537 consensus_queue.cc:260] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.143009 32538 consensus_queue.cc:260] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.143630 32502 heartbeater.cc:503] Master 127.22.121.62:42017 was elected leader, sending a full tablet report...
I20260811 06:17:23.143683 32539 raft_consensus.cc:348] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.144088 32539 raft_consensus.cc:374] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:23.144150 32537 ts_tablet_manager.cc:1428] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 06:17:23.144179 32539 raft_consensus.cc:729] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e14a0a70d4ff41edb74cc1de4bac21de, State: Initialized, Role: FOLLOWER
I20260811 06:17:23.144789 32539 consensus_queue.cc:260] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.145117 32048 heartbeater.cc:503] Master 127.22.121.62:42017 was elected leader, sending a full tablet report...
I20260811 06:17:23.145766 32538 ts_tablet_manager.cc:1428] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Time spent starting tablet: real 0.011s user 0.002s sys 0.010s
I20260811 06:17:23.146885 32282 heartbeater.cc:503] Master 127.22.121.62:42017 was elected leader, sending a full tablet report...
I20260811 06:17:23.147564 32539 ts_tablet_manager.cc:1428] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
W20260811 06:17:23.213181 32050 tablet.cc:2369] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:17:23.288602 32503 tablet.cc:2369] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:17:23.367401 32285 tablet.cc:2369] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:23.389950 32549 raft_consensus.cc:482] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:23.390226 32549 raft_consensus.cc:504] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.391940 32549 leader_election.cc:288] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e14a0a70d4ff41edb74cc1de4bac21de (127.22.121.2:45051), 869020dcaa4346cd86e3e1d6786e46bc (127.22.121.1:46359)
I20260811 06:17:23.394296 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.392120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:45051 (local address 127.22.121.3:35863)
0811 06:17:23.392285 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.392304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.392407 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.392910 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.392914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.392944 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.393528 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.393530 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.393548 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.393598 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.393929 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.393934 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.393940 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.394042 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.394097 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 06:17:23.394341 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.392355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:35863 (local address 127.22.121.2:45051)
0811 06:17:23.392540 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.392544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.392560 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:17:23.392615 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.392619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.392678 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.392774 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.393751 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.393757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.393761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.393780 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.393822 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.393838 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.393937 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.394058 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.394114 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:17:23.395001 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.392598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:57755 (local address 127.22.121.1:46359)
0811 06:17:23.392786 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.392793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.393158 (+ 365us) server_negotiation.cc:407] Connection header received
0811 06:17:23.393282 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.393288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.393337 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.393418 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.394311 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.394316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.394320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.394341 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.394379 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.394395 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.394506 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.394694 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.394753 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:17:23.395130 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.392439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:46359 (local address 127.22.121.3:57755)
0811 06:17:23.393048 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:23.393062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:23.393172 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:23.393456 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:23.393461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:23.393487 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:23.394085 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:17:23.394096 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:23.394111 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:17:23.394154 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:23.394557 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:23.394562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:23.394568 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:17:23.394685 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:17:23.394735 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":323,"threads_started":1}
I20260811 06:17:23.395258 32200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d7e9907707464b9de892d755a2e986" candidate_uuid: "627366f870aa4d18b59947a2a59937b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" is_pre_election: true
I20260811 06:17:23.395794 32200 raft_consensus.cc:2454] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 627366f870aa4d18b59947a2a59937b1 in term 0.
I20260811 06:17:23.395965 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d7e9907707464b9de892d755a2e986" candidate_uuid: "627366f870aa4d18b59947a2a59937b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869020dcaa4346cd86e3e1d6786e46bc" is_pre_election: true
I20260811 06:17:23.396420 32316 leader_election.cc:302] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 627366f870aa4d18b59947a2a59937b1, e14a0a70d4ff41edb74cc1de4bac21de; no voters:
I20260811 06:17:23.396462 31968 raft_consensus.cc:2454] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 627366f870aa4d18b59947a2a59937b1 in term 0.
I20260811 06:17:23.396808 32549 raft_consensus.cc:2781] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:23.396955 32549 raft_consensus.cc:482] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:23.397074 32549 raft_consensus.cc:3037] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:23.399228 32549 raft_consensus.cc:504] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.399824 32549 leader_election.cc:288] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [CANDIDATE]: Term 1 election: Requested vote from peers e14a0a70d4ff41edb74cc1de4bac21de (127.22.121.2:45051), 869020dcaa4346cd86e3e1d6786e46bc (127.22.121.1:46359)
I20260811 06:17:23.400110 32200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d7e9907707464b9de892d755a2e986" candidate_uuid: "627366f870aa4d18b59947a2a59937b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e14a0a70d4ff41edb74cc1de4bac21de"
I20260811 06:17:23.400187 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d7e9907707464b9de892d755a2e986" candidate_uuid: "627366f870aa4d18b59947a2a59937b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869020dcaa4346cd86e3e1d6786e46bc"
I20260811 06:17:23.400274 32200 raft_consensus.cc:3037] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:23.400374 31968 raft_consensus.cc:3037] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:23.402365 32200 raft_consensus.cc:2454] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 627366f870aa4d18b59947a2a59937b1 in term 1.
I20260811 06:17:23.402585 31968 raft_consensus.cc:2454] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 627366f870aa4d18b59947a2a59937b1 in term 1.
I20260811 06:17:23.402681 32316 leader_election.cc:302] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 627366f870aa4d18b59947a2a59937b1, e14a0a70d4ff41edb74cc1de4bac21de; no voters:
I20260811 06:17:23.402961 32549 raft_consensus.cc:2781] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:23.403410 32549 raft_consensus.cc:686] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 1 LEADER]: Becoming Leader. State: Replica: 627366f870aa4d18b59947a2a59937b1, State: Running, Role: LEADER
I20260811 06:17:23.403822 32549 consensus_queue.cc:237] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } }
I20260811 06:17:23.408023 31706 catalog_manager.cc:5495] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 627366f870aa4d18b59947a2a59937b1 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "627366f870aa4d18b59947a2a59937b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "627366f870aa4d18b59947a2a59937b1" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 39051 } health_report { overall_health: HEALTHY } } }
I20260811 06:17:23.459498 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.457643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.22.121.3:39051)
0811 06:17:23.457818 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.457823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.457842 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:17:23.457990 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.457993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.458068 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.458161 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.458915 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.458920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.458926 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.458941 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.458971 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.458983 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.459086 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.459193 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.459251 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:17:23.467110 32200 raft_consensus.cc:1261] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 1 FOLLOWER]: Refusing update from remote peer 627366f870aa4d18b59947a2a59937b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:23.467082 31968 raft_consensus.cc:1261] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 1 FOLLOWER]: Refusing update from remote peer 627366f870aa4d18b59947a2a59937b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:23.468029 32549 consensus_queue.cc:1035] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e14a0a70d4ff41edb74cc1de4bac21de" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:23.468317 32556 consensus_queue.cc:1035] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "869020dcaa4346cd86e3e1d6786e46bc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:23.477618 32564 mvcc.cc:205] Tried to move back new op lower bound from 7317213362031898624 to 7317213361785282560. Current Snapshot: MvccSnapshot[applied={T|T < 7317213362031898624}]
I20260811 06:17:23.479647 32568 mvcc.cc:205] Tried to move back new op lower bound from 7317213362031898624 to 7317213361785282560. Current Snapshot: MvccSnapshot[applied={T|T < 7317213362031898624}]
I20260811 06:17:23.491123 31706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:23.491397 31706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:23.495414 31706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 280592b376de4e42b908a5f8702e1137: Sending DeleteTablet for 3 replicas of tablet f3d7e9907707464b9de892d755a2e986
I20260811 06:17:23.496634 32389 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d7e9907707464b9de892d755a2e986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:23 UTC) from {username='slave'} at 127.0.0.1:36342
I20260811 06:17:23.497524 32578 tablet_replica.cc:330] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: stopping tablet replica
I20260811 06:17:23.497905 32578 raft_consensus.cc:2229] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:23.498479 32578 raft_consensus.cc:2258] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:23.499534 32160 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d7e9907707464b9de892d755a2e986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:23 UTC) from {username='slave'} at 127.0.0.1:60042
I20260811 06:17:23.501052 31938 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d7e9907707464b9de892d755a2e986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:23 UTC) from {username='slave'} at 127.0.0.1:54050
I20260811 06:17:23.501344 32578 ts_tablet_manager.cc:1905] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:23.503943 32581 tablet_replica.cc:330] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: stopping tablet replica
I20260811 06:17:23.504242 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31822
I20260811 06:17:23.504527 32580 tablet_replica.cc:330] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: stopping tablet replica
I20260811 06:17:23.504956 32581 raft_consensus.cc:2229] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:23.505430 32580 raft_consensus.cc:2229] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:23.505927 32581 raft_consensus.cc:2258] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:23.506381 32580 raft_consensus.cc:2258] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:23.507864 32578 ts_tablet_manager.cc:1918] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:23.508025 32578 log.cc:1196] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/f3d7e9907707464b9de892d755a2e986
I20260811 06:17:23.508599 32578 ts_tablet_manager.cc:1939] T f3d7e9907707464b9de892d755a2e986 P 627366f870aa4d18b59947a2a59937b1: Deleting consensus metadata
I20260811 06:17:23.508808 32580 ts_tablet_manager.cc:1905] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:23.510211 31682 catalog_manager.cc:4841] TS 627366f870aa4d18b59947a2a59937b1 (127.22.121.3:39051): tablet f3d7e9907707464b9de892d755a2e986 (table test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]) successfully deleted
I20260811 06:17:23.513947 32580 ts_tablet_manager.cc:1918] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:23.514091 32580 log.cc:1196] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/f3d7e9907707464b9de892d755a2e986
I20260811 06:17:23.514575 32580 ts_tablet_manager.cc:1939] T f3d7e9907707464b9de892d755a2e986 P e14a0a70d4ff41edb74cc1de4bac21de: Deleting consensus metadata
I20260811 06:17:23.515692 32581 ts_tablet_manager.cc:1905] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:23.515868 31687 catalog_manager.cc:4841] TS e14a0a70d4ff41edb74cc1de4bac21de (127.22.121.2:45051): tablet f3d7e9907707464b9de892d755a2e986 (table test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]) successfully deleted
I20260811 06:17:23.522373 32581 ts_tablet_manager.cc:1918] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:23.522526 32581 log.cc:1196] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/f3d7e9907707464b9de892d755a2e986
I20260811 06:17:23.523063 32581 ts_tablet_manager.cc:1939] T f3d7e9907707464b9de892d755a2e986 P 869020dcaa4346cd86e3e1d6786e46bc: Deleting consensus metadata
I20260811 06:17:23.523733 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.505304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.22.121.1:46359)
0811 06:17:23.505469 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.505475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.521766 (+ 16291us) server_negotiation.cc:407] Connection header received
0811 06:17:23.521944 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.521948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.522060 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.522163 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.523008 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.523014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.523018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.523035 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.523067 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.523086 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.523195 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.523370 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.523431 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:23.524341 32037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:23.524681 31687 catalog_manager.cc:4841] TS 869020dcaa4346cd86e3e1d6786e46bc (127.22.121.1:46359): tablet f3d7e9907707464b9de892d755a2e986 (table test-table [id=f61e4d8f83e841fb86ccca1a85af7afa]) successfully deleted
I20260811 06:17:23.671533 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31822
I20260811 06:17:23.698217 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32052
I20260811 06:17:23.701031 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.698972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.22.121.2:45051)
0811 06:17:23.699177 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.699183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.699202 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:17:23.699372 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.699378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.699463 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.699551 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.700421 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.700427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.700431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.700448 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.700482 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.700494 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.700615 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.700701 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.700761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:17:23.701622 32273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:23.810513 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32052
I20260811 06:17:23.843654 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32287
I20260811 06:17:23.846540 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:23.844396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.22.121.3:39051)
0811 06:17:23.844568 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:23.844582 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:23.844601 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:17:23.844824 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:23.844828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:23.844880 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:23.844962 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:23.845899 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:23.845903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:23.845906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:23.845921 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:23.845953 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:23.845964 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:23.846078 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:23.846206 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 06:17:23.846256 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:17:23.847236 32477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:23.962030 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32287
I20260811 06:17:23.992609 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31654
I20260811 06:17:23.993716 31771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:24.096603 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31654
2026-08-11T06:17:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2813 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 06:17:24.143440 23012 mini_oidc.cc:152] Starting JWKS server
I20260811 06:17:24.146282 23012 webserver.cc:460] Webserver started at https://127.0.0.1:33861/ using document root <none> and password file <none>
I20260811 06:17:24.146623 23012 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:17:24.148779 23012 webserver.cc:460] Webserver started at http://127.0.0.1:44383/ using document root <none> and password file <none>
2026-08-11T06:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:24Z Disabled control of system clock
I20260811 06:17:24.174979 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45265
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:45265
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33861/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/testchainca.cert with env {}
W20260811 06:17:24.479179 32598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:24.479465 32598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:24.479600 32598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:24.486480 32598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:24.486578 32598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:24.486652 32598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:24.486739 32598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:24.486795 32598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:17:24.486878 32598 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:17:24.486931 32598 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33861/jwks.json
I20260811 06:17:24.496253 32598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:45265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33861/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:24.497311 32598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:24.498445 32598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:24.509097 32611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:24.509157 32606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:24.509135 32613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:24.509725 32598 server_base.cc:1034] running on GCE node
I20260811 06:17:24.510470 32598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:24.511737 32598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:24.512933 32598 hybrid_clock.cc:648] HybridClock initialized: now 1786429044512888 us; error 41 us; skew 500 ppm
I20260811 06:17:24.513238 32598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:24.515400 32598 webserver.cc:460] Webserver started at http://127.22.121.62:44681/ using document root <none> and password file <none>
I20260811 06:17:24.515993 32598 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:24.516088 32598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:24.516355 32598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:24.518914 32598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "2a21de6f204445ffae644a9ebada8286"
format_stamp: "Formatted at 2026-08-11 06:17:24 on dist-test-slave-gjrz"
server_key: "5d1a44c04eba22c4ee54548eb94bbcc9"
server_key_iv: "a94a430d70f5d548e417d1663bad6a01"
server_key_version: "encryptionkey@0"
I20260811 06:17:24.519721 32598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "2a21de6f204445ffae644a9ebada8286"
format_stamp: "Formatted at 2026-08-11 06:17:24 on dist-test-slave-gjrz"
server_key: "5d1a44c04eba22c4ee54548eb94bbcc9"
server_key_iv: "a94a430d70f5d548e417d1663bad6a01"
server_key_version: "encryptionkey@0"
I20260811 06:17:24.524394 32598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 06:17:24.527436 32624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:24.528611 32598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:17:24.528754 32598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "2a21de6f204445ffae644a9ebada8286"
format_stamp: "Formatted at 2026-08-11 06:17:24 on dist-test-slave-gjrz"
server_key: "5d1a44c04eba22c4ee54548eb94bbcc9"
server_key_iv: "a94a430d70f5d548e417d1663bad6a01"
server_key_version: "encryptionkey@0"
I20260811 06:17:24.528934 32598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:24.552605 32598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:24.553259 32598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:24.553522 32598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:24.569731 32598 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:45265
I20260811 06:17:24.569762 32708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:45265 every 8 connection(s)
I20260811 06:17:24.570859 32598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:24.574244 32709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:24.578598 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32598
I20260811 06:17:24.578792 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:24.579608 23012 external_mini_cluster.cc:1428] Setting key 77306eea649008eec47e7ea4936196e3
I20260811 06:17:24.585096 32709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286: Bootstrap starting.
I20260811 06:17:24.588397 32709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:24.589493 32709 log.cc:824] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:24.592067 32709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286: No bootstrap required, opened a new log
I20260811 06:17:24.592418 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:24.581349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.22.121.62:45265)
0811 06:17:24.582269 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:24.582279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:24.582319 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:17:24.584044 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:24.584066 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:24.584461 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:24.584741 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:24.585351 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.586605 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.588386 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.589044 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.589179 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:24.590359 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:24.590366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:24.590383 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:24.590850 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:24.590921 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:24.590941 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:24.591044 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:24.591254 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 06:17:24.591542 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":335,"threads_started":1}
I20260811 06:17:24.595999 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:17:24.600189 32709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } }
I20260811 06:17:24.600476 32709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:24.600567 32709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a21de6f204445ffae644a9ebada8286, State: Initialized, Role: FOLLOWER
I20260811 06:17:24.601063 32709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } }
I20260811 06:17:24.601261 32709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:24.601382 32709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:24.601532 32709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:24.603719 32709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } }
I20260811 06:17:24.604235 32709 leader_election.cc:302] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a21de6f204445ffae644a9ebada8286; no voters:
I20260811 06:17:24.604724 32709 leader_election.cc:288] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:24.604851 32722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:24.605487 32722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [term 1 LEADER]: Becoming Leader. State: Replica: 2a21de6f204445ffae644a9ebada8286, State: Running, Role: LEADER
I20260811 06:17:24.605962 32722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } }
I20260811 06:17:24.606452 32709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:24.609338 32723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a21de6f204445ffae644a9ebada8286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } } }
I20260811 06:17:24.609746 32723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:24.610314 32724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a21de6f204445ffae644a9ebada8286. Latest consensus state: current_term: 1 leader_uuid: "2a21de6f204445ffae644a9ebada8286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21de6f204445ffae644a9ebada8286" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45265 } } }
I20260811 06:17:24.610664 32724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:24.610926 32739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:24.615291 32739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:24.622308 32739 catalog_manager.cc:1347] Generated new cluster ID: 5a973499fbd541f49f9ac661af5d3430
I20260811 06:17:24.622406 32739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:24.629961 32739 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:24.631573 32739 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:24.640041 32739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a21de6f204445ffae644a9ebada8286: Generated new TSK 0
I20260811 06:17:24.640869 32739 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:17:24.675199 32765 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 06:17:24.675426 32765 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:17:24.677650 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:24.655579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.22.121.62:45265)
0811 06:17:24.655773 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:24.655780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:24.656235 (+ 455us) server_negotiation.cc:407] Connection header received
0811 06:17:24.656610 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:24.656613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:24.656691 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:24.656801 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:17:24.657421 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.658106 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.660133 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.660544 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.660653 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:24.661216 (+ 563us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:17:24.677178 (+ 15962us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:17:24.677438 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:17:24.677758 32715 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 06:17:24.679886 32764 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:24.655494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45265 (local address 127.0.0.1:35998)
0811 06:17:24.656126 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:24.656150 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:24.656333 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:24.656850 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:24.656856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:24.656869 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:17:24.657265 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:24.657278 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.658240 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.658246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:24.659996 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:24.660012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:24.660733 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:24.660740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:24.660922 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:24.660998 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:17:24.677417 (+ 16419us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:17:24.677723 (+ 306us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.121.62:45265: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":299,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.121.62:45265: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33861/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 06:17:24.681794 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32598
I20260811 06:17:24.682766 32702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:24.771873 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32598
2026-08-11T06:17:24Z chronyd exiting
I20260811 06:17:24.829807 23012 test_util.cc:175] -----------------------------------------------
I20260811 06:17:24.829921 23012 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429000386903-23012-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (691 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 06:17:24.833521 23012 mini_oidc.cc:152] Starting JWKS server
I20260811 06:17:24.835180 23012 webserver.cc:460] Webserver started at https://127.0.0.1:41889/ using document root <none> and password file <none>
I20260811 06:17:24.835424 23012 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:17:24.836009 23012 webserver.cc:460] Webserver started at http://127.0.0.1:34957/ using document root <none> and password file <none>
2026-08-11T06:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:24Z Disabled control of system clock
I20260811 06:17:24.862434 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42299
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:42299
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41889/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/testchainca.cert with env {}
W20260811 06:17:25.167459 312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:25.167717 312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:25.167857 312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:25.174762 312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:25.174866 312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:25.174940 312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:25.175004 312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:25.175046 312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:17:25.175128 312 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:17:25.175184 312 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41889/jwks.json
I20260811 06:17:25.184741 312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:42299
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41889/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:25.185951 312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:25.187201 312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:25.198238 326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:25.198300 327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:25.198343 329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:25.199083 312 server_base.cc:1034] running on GCE node
I20260811 06:17:25.199852 312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:25.201177 312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:25.202394 312 hybrid_clock.cc:648] HybridClock initialized: now 1786429045202343 us; error 38 us; skew 500 ppm
I20260811 06:17:25.202746 312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:25.204922 312 webserver.cc:460] Webserver started at http://127.22.121.62:33037/ using document root <none> and password file <none>
I20260811 06:17:25.205585 312 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:25.205685 312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:25.205997 312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:25.208803 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "10182cf7f3ead8efbdda2127bcf1ecea"
server_key_iv: "a164001afb61ee5eed657e1bb779eaf9"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.209560 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "10182cf7f3ead8efbdda2127bcf1ecea"
server_key_iv: "a164001afb61ee5eed657e1bb779eaf9"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.214318 312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 06:17:25.217461 337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:25.218624 312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:17:25.218780 312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "10182cf7f3ead8efbdda2127bcf1ecea"
server_key_iv: "a164001afb61ee5eed657e1bb779eaf9"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.218974 312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:25.268688 312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:25.269496 312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:25.269786 312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:25.286401 312 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:42299
I20260811 06:17:25.286434 421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:42299 every 8 connection(s)
I20260811 06:17:25.287642 312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:25.291239 422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:25.296710 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 312
I20260811 06:17:25.296904 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:25.297533 23012 external_mini_cluster.cc:1428] Setting key 3a3206ddd9c0f2c597f00b0d96dbc6c0
I20260811 06:17:25.302093 422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9: Bootstrap starting.
I20260811 06:17:25.305994 422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:25.307126 422 log.cc:824] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:25.310132 422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9: No bootstrap required, opened a new log
I20260811 06:17:25.312530 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:25.299460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.22.121.62:42299)
0811 06:17:25.300471 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:25.300486 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:25.300546 (+ 60us) server_negotiation.cc:407] Connection header received
0811 06:17:25.302849 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:25.302877 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:25.303438 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:25.303891 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:25.304459 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.306441 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.307824 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.308657 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.308826 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:25.309870 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:25.309880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:25.309897 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:25.310582 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:25.310687 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:25.310728 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:25.310845 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:25.311003 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 06:17:25.311321 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":313,"threads_started":1}
I20260811 06:17:25.315969 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:17:25.318231 422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } }
I20260811 06:17:25.318534 422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:25.318634 422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644d12a8ecdc4c1cbdf1e10bd4ebf3b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:25.319218 422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } }
I20260811 06:17:25.319473 422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:25.319609 422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:25.319764 422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:25.322041 422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } }
I20260811 06:17:25.322544 422 leader_election.cc:302] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 644d12a8ecdc4c1cbdf1e10bd4ebf3b9; no voters:
I20260811 06:17:25.323108 422 leader_election.cc:288] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:25.323195 431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:25.323853 431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [term 1 LEADER]: Becoming Leader. State: Replica: 644d12a8ecdc4c1cbdf1e10bd4ebf3b9, State: Running, Role: LEADER
I20260811 06:17:25.324335 431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } }
I20260811 06:17:25.324935 422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:25.326877 432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } } }
I20260811 06:17:25.326849 433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 644d12a8ecdc4c1cbdf1e10bd4ebf3b9. Latest consensus state: current_term: 1 leader_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644d12a8ecdc4c1cbdf1e10bd4ebf3b9" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42299 } } }
I20260811 06:17:25.327257 432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:25.327320 433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:25.331758 447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:25.335563 447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:25.343184 447 catalog_manager.cc:1347] Generated new cluster ID: f8513acfca66452baae0fba4dc8c910e
I20260811 06:17:25.343297 447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:25.364949 447 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:25.366070 447 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:25.374174 447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 644d12a8ecdc4c1cbdf1e10bd4ebf3b9: Generated new TSK 0
I20260811 06:17:25.374819 447 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:17:25.409905 469 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 06:17:25.410022 469 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:17:25.410876 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:25.391928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.22.121.62:42299)
0811 06:17:25.392127 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:25.392134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:25.392657 (+ 523us) server_negotiation.cc:407] Connection header received
0811 06:17:25.392879 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:25.392888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:25.392974 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:25.393123 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:17:25.393625 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.394342 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.396304 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.396734 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.396826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:25.397332 (+ 506us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:17:25.410436 (+ 13104us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:17:25.410688 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 06:17:25.410975 429 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 06:17:25.411257 468 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:25.391928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:42299 (local address 127.0.0.1:60418)
0811 06:17:25.392545 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:25.392565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:25.392709 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:25.393144 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:25.393150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:25.393162 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:17:25.393481 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:25.393493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.394471 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.394477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:25.396176 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:25.396192 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:25.396878 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:25.396884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:25.397078 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:25.397122 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:17:25.410629 (+ 13507us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:17:25.410873 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.121.62:42299: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":306,"threads_started":1}
I20260811 06:17:25.412886 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 312
I20260811 06:17:25.413780 417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:25.504792 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 312
2026-08-11T06:17:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (734 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 06:17:25.568110 23012 mini_oidc.cc:152] Starting JWKS server
I20260811 06:17:25.569926 23012 webserver.cc:460] Webserver started at https://127.0.0.1:42891/ using document root <none> and password file <none>
I20260811 06:17:25.570139 23012 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:17:25.570724 23012 webserver.cc:460] Webserver started at http://127.0.0.1:38513/ using document root <none> and password file <none>
2026-08-11T06:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:25Z Disabled control of system clock
I20260811 06:17:25.595088 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38125
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:38727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38125
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42891/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:17:25.897256 479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:25.897526 479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:25.897658 479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:25.904580 479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:25.904682 479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:25.904757 479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:25.904834 479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:25.904894 479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:17:25.904976 479 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:17:25.905030 479 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42891/jwks.json
I20260811 06:17:25.913597 479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42891/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:25.914700 479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:25.916606 479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:25.927665 491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:25.927587 501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:25.928098 495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:25.928608 479 server_base.cc:1034] running on GCE node
I20260811 06:17:25.929330 479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:25.930624 479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:25.931826 479 hybrid_clock.cc:648] HybridClock initialized: now 1786429045931781 us; error 30 us; skew 500 ppm
I20260811 06:17:25.932168 479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:25.934247 479 webserver.cc:460] Webserver started at http://127.22.121.62:39669/ using document root <none> and password file <none>
I20260811 06:17:25.934871 479 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:25.934984 479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:25.935323 479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:25.938175 479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "ff02778665c84c789880e6e85975727a"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "111b7cea311736f450f3fb706297c65e"
server_key_iv: "6d553681d2ece4b6de77d404f4600336"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.938935 479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "ff02778665c84c789880e6e85975727a"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "111b7cea311736f450f3fb706297c65e"
server_key_iv: "6d553681d2ece4b6de77d404f4600336"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.943542 479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260811 06:17:25.946658 511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:25.947832 479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:25.947992 479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "ff02778665c84c789880e6e85975727a"
format_stamp: "Formatted at 2026-08-11 06:17:25 on dist-test-slave-gjrz"
server_key: "111b7cea311736f450f3fb706297c65e"
server_key_iv: "6d553681d2ece4b6de77d404f4600336"
server_key_version: "encryptionkey@0"
I20260811 06:17:25.948174 479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:25.997006 479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:25.997761 479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:25.998047 479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:26.014499 479 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:38125
I20260811 06:17:26.014518 596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:38125 every 8 connection(s)
I20260811 06:17:26.015726 479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:26.018911 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 479
I20260811 06:17:26.019101 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:26.019620 597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:26.019940 23012 external_mini_cluster.cc:1428] Setting key 3b3156c01b3d1cde7ad9d15a48bdec74
I20260811 06:17:26.032531 597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a: Bootstrap starting.
I20260811 06:17:26.032709 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:26.021680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.22.121.62:38125)
0811 06:17:26.022684 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:26.022703 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:26.022743 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:17:26.024213 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:26.024234 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:26.024665 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:26.024964 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:26.025508 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.026796 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.028545 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.029214 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.029368 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.030555 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:26.030562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:26.030582 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:26.031085 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:26.031171 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:26.031191 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:26.031310 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:26.031509 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 06:17:26.031806 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":407,"threads_started":1}
I20260811 06:17:26.035490 597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:26.036082 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:17:26.036567 597 log.cc:824] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:26.039307 597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a: No bootstrap required, opened a new log
I20260811 06:17:26.045892 597 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } }
I20260811 06:17:26.046240 597 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:26.046372 597 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff02778665c84c789880e6e85975727a, State: Initialized, Role: FOLLOWER
I20260811 06:17:26.046970 597 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } }
I20260811 06:17:26.047256 597 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:26.047426 597 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:26.047610 597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:26.050077 597 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } }
I20260811 06:17:26.050491 597 leader_election.cc:302] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff02778665c84c789880e6e85975727a; no voters:
I20260811 06:17:26.050997 597 leader_election.cc:288] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:26.051136 607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:26.051820 607 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [term 1 LEADER]: Becoming Leader. State: Replica: ff02778665c84c789880e6e85975727a, State: Running, Role: LEADER
I20260811 06:17:26.052347 607 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } }
I20260811 06:17:26.052955 597 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:26.055094 611 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff02778665c84c789880e6e85975727a. Latest consensus state: current_term: 1 leader_uuid: "ff02778665c84c789880e6e85975727a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } } }
I20260811 06:17:26.055119 609 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff02778665c84c789880e6e85975727a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff02778665c84c789880e6e85975727a" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38125 } } }
I20260811 06:17:26.055514 611 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:26.055514 609 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:26.057860 616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:26.062443 616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:26.070811 616 catalog_manager.cc:1347] Generated new cluster ID: 311c0a259fb844669c2407387962ace0
I20260811 06:17:26.070899 616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:26.092823 616 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:26.093927 616 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:26.099311 616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff02778665c84c789880e6e85975727a: Generated new TSK 0
I20260811 06:17:26.100039 616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:17:26.131366 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:26.124591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.22.121.62:38125)
0811 06:17:26.124733 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:26.124737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:26.125084 (+ 347us) server_negotiation.cc:407] Connection header received
0811 06:17:26.125302 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:26.125306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:26.125369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:26.125491 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:17:26.125998 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.126688 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.128034 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.128507 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.128621 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.129078 (+ 457us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:17:26.130767 (+ 1689us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42891/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:17:26.131045 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42891/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 06:17:26.131484 605 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42891/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 06:17:26.131556 654 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:26.124451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38125 (local address 127.0.0.1:38832)
0811 06:17:26.124991 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:26.125006 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:26.125137 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:26.125528 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:26.125532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:26.125544 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:17:26.125859 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:26.125868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.126824 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.126830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:26.127902 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:26.127914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.128691 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.128698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:26.128827 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.128858 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:17:26.130946 (+ 2088us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42891/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:17:26.131161 (+ 215us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.121.62:38125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42891/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":286,"threads_started":1}
I20260811 06:17:26.132517 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 479
I20260811 06:17:26.133550 592 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 06:17:26.143487 655 webserver.cc:533] Webserver: sslize failed:
E20260811 06:17:26.143574 655 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:17:26.222606 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 479
2026-08-11T06:17:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (715 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T06:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:26Z Disabled control of system clock
I20260811 06:17:26.312222 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45213
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:45213
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/testchainca.cert with env {}
W20260811 06:17:26.617357 666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:26.617611 666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:26.617697 666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:26.624504 666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:26.624593 666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:26.624636 666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:26.624694 666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:26.624737 666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:17:26.624794 666 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:17:26.624832 666 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 06:17:26.633986 666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:45213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:26.635023 666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:26.636158 666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:26.647014 677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:26.647079 674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:26.647132 679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:26.647650 666 server_base.cc:1034] running on GCE node
I20260811 06:17:26.648425 666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:26.649641 666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:26.650831 666 hybrid_clock.cc:648] HybridClock initialized: now 1786429046650790 us; error 31 us; skew 500 ppm
I20260811 06:17:26.651124 666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:17:26.653105 666 webserver.cc:460] Webserver started at http://127.22.121.62:44123/ using document root <none> and password file <none>
I20260811 06:17:26.653697 666 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:26.653793 666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:26.654085 666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:26.656736 666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "f3782ff4435044c8ab34f5f6dc3676cc"
format_stamp: "Formatted at 2026-08-11 06:17:26 on dist-test-slave-gjrz"
server_key: "7ccc3b233f97ef91a6d2bf3cede687ad"
server_key_iv: "da00522807e7736818b964de81b47943"
server_key_version: "encryptionkey@0"
I20260811 06:17:26.657524 666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "f3782ff4435044c8ab34f5f6dc3676cc"
format_stamp: "Formatted at 2026-08-11 06:17:26 on dist-test-slave-gjrz"
server_key: "7ccc3b233f97ef91a6d2bf3cede687ad"
server_key_iv: "da00522807e7736818b964de81b47943"
server_key_version: "encryptionkey@0"
I20260811 06:17:26.662336 666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:17:26.665416 692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:26.666597 666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:17:26.666749 666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "f3782ff4435044c8ab34f5f6dc3676cc"
format_stamp: "Formatted at 2026-08-11 06:17:26 on dist-test-slave-gjrz"
server_key: "7ccc3b233f97ef91a6d2bf3cede687ad"
server_key_iv: "da00522807e7736818b964de81b47943"
server_key_version: "encryptionkey@0"
I20260811 06:17:26.666925 666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:26.691803 666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:26.692509 666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:26.692754 666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:26.709730 666 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:45213
I20260811 06:17:26.709822 783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:45213 every 8 connection(s)
I20260811 06:17:26.710978 666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:26.714474 787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:26.717679 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 666
I20260811 06:17:26.717906 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:26.718747 23012 external_mini_cluster.cc:1428] Setting key 56e6110915bdc5bb8cf89516c7ccad87
I20260811 06:17:26.726980 787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc: Bootstrap starting.
I20260811 06:17:26.730337 787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:26.731478 787 log.cc:824] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:26.732074 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:26.720669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.22.121.62:45213)
0811 06:17:26.721611 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:26.721621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:26.721660 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:17:26.723216 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:26.723236 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:26.723719 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:26.724037 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:26.724608 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.726110 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.727850 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.728546 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.728684 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.729808 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:26.729816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:26.729829 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:17:26.730318 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:26.730414 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:17:26.730433 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:26.730548 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:26.730760 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:17:26.731050 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":339,"threads_started":1}
I20260811 06:17:26.734309 787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc: No bootstrap required, opened a new log
I20260811 06:17:26.735754 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:17:26.742465 787 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } }
I20260811 06:17:26.742751 787 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:26.742820 787 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3782ff4435044c8ab34f5f6dc3676cc, State: Initialized, Role: FOLLOWER
I20260811 06:17:26.743402 787 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } }
I20260811 06:17:26.743613 787 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:26.743767 787 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:26.743952 787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:26.745862 787 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } }
I20260811 06:17:26.746304 787 leader_election.cc:302] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3782ff4435044c8ab34f5f6dc3676cc; no voters:
I20260811 06:17:26.746788 787 leader_election.cc:288] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:26.746935 796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:26.748225 796 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [term 1 LEADER]: Becoming Leader. State: Replica: f3782ff4435044c8ab34f5f6dc3676cc, State: Running, Role: LEADER
I20260811 06:17:26.748724 796 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } }
I20260811 06:17:26.748958 787 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:26.751324 802 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3782ff4435044c8ab34f5f6dc3676cc. Latest consensus state: current_term: 1 leader_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } } }
I20260811 06:17:26.751300 798 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3782ff4435044c8ab34f5f6dc3676cc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45213 } } }
I20260811 06:17:26.751724 798 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:26.751724 802 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:26.752995 812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:26.757965 812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:26.765985 812 catalog_manager.cc:1347] Generated new cluster ID: 2ad08aaff8704e58b7c36b5fabb0964f
I20260811 06:17:26.766091 812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:26.782330 812 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:26.783524 812 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:26.800745 812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3782ff4435044c8ab34f5f6dc3676cc: Generated new TSK 0
I20260811 06:17:26.801466 812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:17:27.696187 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:26.820206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.22.121.62:45213)
0811 06:17:26.820372 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:26.820378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:26.820732 (+ 354us) server_negotiation.cc:407] Connection header received
0811 06:17:26.820936 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:26.820940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:26.821011 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:26.821122 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:17:26.821601 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.822301 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.824255 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.824702 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.824788 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.825338 (+ 550us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:17:27.695588 (+870250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:17:27.695926 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37934: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 06:17:27.696280 793 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37934: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 06:17:27.696560 839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:26.820051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45213 (local address 127.0.0.1:37934)
0811 06:17:26.820633 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:26.820651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:26.820794 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:26.821152 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:26.821157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:26.821170 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:17:26.821456 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:26.821468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.822427 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.822433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:26.824122 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:26.824138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:26.824854 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:26.824860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:26.825049 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:26.825103 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:17:27.695839 (+870736us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:17:27.696185 (+ 346us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.121.62:45213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":275,"threads_started":1}
I20260811 06:17:27.697782 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 666
I20260811 06:17:27.698850 776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:27.797462 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 666
2026-08-11T06:17:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1556 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (398 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (390 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:28 dist-test-slave-gjrz krb5kdc[864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:28 dist-test-slave-gjrz krb5kdc[864](info): set up 2 sockets
Aug 11 06:17:28 dist-test-slave-gjrz krb5kdc[864](info): commencing operation
krb5kdc: starting...
W20260811 06:17:29.813360 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:29 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:30.004789 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33955
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:33955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:30.315486 896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:30.315737 896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:30.315822 896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:30.322625 896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:30.322714 896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:30.322763 896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:30.322821 896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:30.322863 896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:30.332023 896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:33955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:30.333120 896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:30.334285 896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:30.345448 908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:30.345458 906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:30.345721 905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:30.346185 896 server_base.cc:1034] running on GCE node
I20260811 06:17:30.346863 896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:30.348160 896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:30.349367 896 hybrid_clock.cc:648] HybridClock initialized: now 1786429050349314 us; error 49 us; skew 500 ppm
I20260811 06:17:30.349711 896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:30 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429050, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:30.357422 896 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:30.358754 896 webserver.cc:460] Webserver started at http://127.22.121.62:43241/ using document root <none> and password file <none>
I20260811 06:17:30.359400 896 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:30.359503 896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:30.359750 896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:30.362504 896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "96989b0d475c4ce5994fbff7343beadc"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "29bdff4651473179e4a157be2d421f94"
server_key_iv: "44ee46e721328b8a4e356a5644b057b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.363200 896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "96989b0d475c4ce5994fbff7343beadc"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "29bdff4651473179e4a157be2d421f94"
server_key_iv: "44ee46e721328b8a4e356a5644b057b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.367947 896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:17:30.371016 918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:30.372218 896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:17:30.372370 896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "96989b0d475c4ce5994fbff7343beadc"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "29bdff4651473179e4a157be2d421f94"
server_key_iv: "44ee46e721328b8a4e356a5644b057b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.372536 896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:30.388907 896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:30.392294 896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:30.392529 896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:30.408885 896 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:33955
I20260811 06:17:30.408942 1001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:33955 every 8 connection(s)
I20260811 06:17:30.410059 896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:30.410768 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 896
I20260811 06:17:30.410998 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:30.411689 23012 external_mini_cluster.cc:1428] Setting key 0397d56c7b6d1b53ce8b7d94076835be
I20260811 06:17:30.413991 1002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:17:30 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:30.426129 1002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc: Bootstrap starting.
I20260811 06:17:30.429956 1002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:30.431119 1002 log.cc:824] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:30.434235 1002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc: No bootstrap required, opened a new log
I20260811 06:17:30.434397 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:30.413481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.22.121.62:33955)
0811 06:17:30.414271 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:30.414282 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:30.414321 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:17:30.415956 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:30.415977 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:30.416336 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:30.416656 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:30.418406 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:30.419771 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:30.421410 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:30.422057 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:30.422186 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:30.426498 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:30.426508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:30.426525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:30.426570 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:30.430224 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:30.430875 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:30.430883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:30.430891 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:30.431085 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:30.431486 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:30.431491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:30.431494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:30.432065 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:30.432284 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:30.432787 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 06:17:30.433180 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":344,"threads_started":1}
I20260811 06:17:30.441992 1002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } }
I20260811 06:17:30.442298 1002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:30.442370 1002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96989b0d475c4ce5994fbff7343beadc, State: Initialized, Role: FOLLOWER
I20260811 06:17:30.442816 1002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } }
I20260811 06:17:30.443032 1002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:30.443161 1002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:30.443342 1002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:30.445287 1002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } }
I20260811 06:17:30.445801 1002 leader_election.cc:302] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96989b0d475c4ce5994fbff7343beadc; no voters:
I20260811 06:17:30.446422 1002 leader_election.cc:288] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:30.446511 1019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:30.447149 1019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [term 1 LEADER]: Becoming Leader. State: Replica: 96989b0d475c4ce5994fbff7343beadc, State: Running, Role: LEADER
I20260811 06:17:30.447685 1019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } }
I20260811 06:17:30.448376 1002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:30.450434 1020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96989b0d475c4ce5994fbff7343beadc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } } }
I20260811 06:17:30.450579 1021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96989b0d475c4ce5994fbff7343beadc. Latest consensus state: current_term: 1 leader_uuid: "96989b0d475c4ce5994fbff7343beadc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96989b0d475c4ce5994fbff7343beadc" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33955 } } }
I20260811 06:17:30.450817 1021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:30.450821 1020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:30.452558 1038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:30.456914 1038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:30.465008 1038 catalog_manager.cc:1347] Generated new cluster ID: 58b44c404b6d410bbb6be52d95d91117
I20260811 06:17:30.465113 1038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:30.483918 1038 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:30.485173 1038 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:30.496124 1038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96989b0d475c4ce5994fbff7343beadc: Generated new TSK 0
I20260811 06:17:30.497030 1038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:30.555238 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:30.866530 1060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:30.866781 1060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:30.866909 1060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:30.873929 1060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:30.874025 1060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:30.874169 1060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:30.883656 1060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:30.884847 1060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:30.886094 1060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:30.898301 1077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:30.898350 1080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:30.898377 1074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:30.899333 1060 server_base.cc:1034] running on GCE node
I20260811 06:17:30.899873 1060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:30.900717 1060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:30.901923 1060 hybrid_clock.cc:648] HybridClock initialized: now 1786429050901866 us; error 38 us; skew 500 ppm
I20260811 06:17:30.902210 1060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:30 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429050, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:30.905261 1060 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:30.906620 1060 webserver.cc:460] Webserver started at http://127.22.121.1:33007/ using document root <none> and password file <none>
I20260811 06:17:30.907241 1060 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:30.907397 1060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:30.907655 1060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:30.910406 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "87eb489ddc8b40b989a3ed7f18db0ad5"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "3edbb1f1e638a19a3da5ef31e72309b8"
server_key_iv: "f7d1eca8911629c36effd1ab2371d9c7"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.911135 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "87eb489ddc8b40b989a3ed7f18db0ad5"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "3edbb1f1e638a19a3da5ef31e72309b8"
server_key_iv: "f7d1eca8911629c36effd1ab2371d9c7"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.915942 1060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:17:30.919076 1093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:30.920457 1060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:17:30.920609 1060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "87eb489ddc8b40b989a3ed7f18db0ad5"
format_stamp: "Formatted at 2026-08-11 06:17:30 on dist-test-slave-gjrz"
server_key: "3edbb1f1e638a19a3da5ef31e72309b8"
server_key_iv: "f7d1eca8911629c36effd1ab2371d9c7"
server_key_version: "encryptionkey@0"
I20260811 06:17:30.920794 1060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:30.957499 1060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:30.961597 1060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:30.961856 1060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:30.963186 1060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:30.964490 1060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:30.964564 1060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:30.964689 1060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:30.964743 1060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:30.994696 1060 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:40563
I20260811 06:17:30.994760 1307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:40563 every 8 connection(s)
I20260811 06:17:30.996110 1060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:31.003129 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 1060
I20260811 06:17:31.003360 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:31.004177 23012 external_mini_cluster.cc:1428] Setting key 14f19bdbcc128bb0178fc51bcd092392
Aug 11 06:17:31 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429050, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:31.016000 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:30.998574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:35165 (local address 127.22.121.62:33955)
0811 06:17:30.998988 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:30.998993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:30.999972 (+ 979us) server_negotiation.cc:407] Connection header received
0811 06:17:31.001732 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:31.001738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:31.001819 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:31.001928 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:31.003399 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.004490 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.006167 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.006742 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.006879 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.010409 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:31.010415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:31.010418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:31.010451 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:31.012858 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.013675 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.013679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.013681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.013798 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.014259 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.014263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.014266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.014588 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:31.014737 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:31.015434 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 06:17:31.015668 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":198,"threads_started":1}
I20260811 06:17:31.016719 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:30.999080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33955 (local address 127.22.121.1:35165)
0811 06:17:30.999807 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:30.999842 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:31.001470 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:31.002169 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:31.002177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:31.002648 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:31.003211 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.003229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.004628 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.004632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.006020 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.006035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.006938 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.006944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.007146 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.007790 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:31.007816 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:31.010198 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:31.013001 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.013010 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.013025 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.013567 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.013893 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.013897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.013900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.014143 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.014712 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:31.014718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:31.015020 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 06:17:31.015286 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 06:17:31.015679 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":334,"threads_started":1}
I20260811 06:17:31.018158 1308 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33955
I20260811 06:17:31.018456 1308 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:31.019364 1308 heartbeater.cc:511] Master 127.22.121.62:33955 requested a full tablet report, sending...
I20260811 06:17:31.021447 942 ts_manager.cc:194] Registered new tserver with Master: 87eb489ddc8b40b989a3ed7f18db0ad5 (127.22.121.1:40563)
I20260811 06:17:31.022919 942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:35165
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:31.052639 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:31.365818 1320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:31.366078 1320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:31.366200 1320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:31.373117 1320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:31.373208 1320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:31.373348 1320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:31.382421 1320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:31.383553 1320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:31.384760 1320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:31.397075 1333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:31.397114 1334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:31.397161 1336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:31.398053 1320 server_base.cc:1034] running on GCE node
I20260811 06:17:31.398643 1320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:31.399493 1320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:31.400692 1320 hybrid_clock.cc:648] HybridClock initialized: now 1786429051400659 us; error 28 us; skew 500 ppm
I20260811 06:17:31.400969 1320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:31 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429051, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:31.404035 1320 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:31.405296 1320 webserver.cc:460] Webserver started at http://127.22.121.2:36261/ using document root <none> and password file <none>
I20260811 06:17:31.405876 1320 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:31.405980 1320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:31.406221 1320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:31.409287 1320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "d598d1f5b84d47c9bfaf09001077aef9"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "2744b5aafe98fbf0f2f24e15d836067c"
server_key_iv: "378e20e30ca4b78cd8292fce9d058258"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.410241 1320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "d598d1f5b84d47c9bfaf09001077aef9"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "2744b5aafe98fbf0f2f24e15d836067c"
server_key_iv: "378e20e30ca4b78cd8292fce9d058258"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.416244 1320 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 06:17:31.420171 1348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.421550 1320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 06:17:31.421703 1320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "d598d1f5b84d47c9bfaf09001077aef9"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "2744b5aafe98fbf0f2f24e15d836067c"
server_key_iv: "378e20e30ca4b78cd8292fce9d058258"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.421919 1320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:31.446587 1320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:31.450791 1320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:31.451028 1320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:31.451926 1320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:31.453150 1320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:31.453219 1320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.453303 1320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:31.453346 1320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.482657 1320 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:34377
I20260811 06:17:31.482718 1567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:34377 every 8 connection(s)
I20260811 06:17:31.483884 1320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:31.489941 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 1320
I20260811 06:17:31.490139 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:31.490554 23012 external_mini_cluster.cc:1428] Setting key 0d6e9f80d4b2d1dad8d8643ff21c2c56
Aug 11 06:17:31 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429051, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:31.505940 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:31.486316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:59449 (local address 127.22.121.62:33955)
0811 06:17:31.486489 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:31.486493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:31.487745 (+ 1252us) server_negotiation.cc:407] Connection header received
0811 06:17:31.489852 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:31.489858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:31.489936 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:31.490024 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:31.491783 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.493038 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.495025 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.495643 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.495753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.499993 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:31.499999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:31.500003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:31.500038 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:31.502860 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.503565 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.503571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.503574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.503721 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.504221 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.504224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.504226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.504477 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:31.504581 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:31.505403 (+ 822us) server_negotiation.cc:299] Negotiation successful
0811 06:17:31.505621 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:17:31.506626 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:31.486745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33955 (local address 127.22.121.2:59449)
0811 06:17:31.487522 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:31.487590 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:31.489574 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:31.490273 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:31.490283 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:31.490832 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:31.491600 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.491624 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.493192 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.493197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.494863 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.494878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.495920 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.495930 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.496237 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.497183 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:31.497212 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:31.499774 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:31.502989 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.502995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.503006 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.503443 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.503845 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.503849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.503853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.504120 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.504599 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:31.504606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:31.504988 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 06:17:31.505265 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 06:17:31.505611 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":286,"threads_started":1}
I20260811 06:17:31.508002 1569 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33955
I20260811 06:17:31.508272 1569 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:31.508908 1569 heartbeater.cc:511] Master 127.22.121.62:33955 requested a full tablet report, sending...
I20260811 06:17:31.510250 942 ts_manager.cc:194] Registered new tserver with Master: d598d1f5b84d47c9bfaf09001077aef9 (127.22.121.2:34377)
I20260811 06:17:31.511019 942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:59449
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:31.540096 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:31.851406 1576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:31.851668 1576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:31.851790 1576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:31.858690 1576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:31.858796 1576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:31.858951 1576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:31.868040 1576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:31.869145 1576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:31.870357 1576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:31.882577 1586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:31.882550 1583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:31.883117 1582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:31.883230 1576 server_base.cc:1034] running on GCE node
I20260811 06:17:31.883955 1576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:31.884752 1576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:31.885948 1576 hybrid_clock.cc:648] HybridClock initialized: now 1786429051885880 us; error 56 us; skew 500 ppm
I20260811 06:17:31.886212 1576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:31 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429051, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:31.889194 1576 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:31.890600 1576 webserver.cc:460] Webserver started at http://127.22.121.3:38227/ using document root <none> and password file <none>
I20260811 06:17:31.891202 1576 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:31.891341 1576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:31.891652 1576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:31.894438 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "708b0db84b23404898be6c51a7c32708"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "0843f47d513874658fd600fc23aae749"
server_key_iv: "483e031719b62333f1cd1bad50c72bbf"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.895211 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "708b0db84b23404898be6c51a7c32708"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "0843f47d513874658fd600fc23aae749"
server_key_iv: "483e031719b62333f1cd1bad50c72bbf"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.900009 1576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:17:31.903285 1607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.904492 1576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:17:31.904626 1576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "708b0db84b23404898be6c51a7c32708"
format_stamp: "Formatted at 2026-08-11 06:17:31 on dist-test-slave-gjrz"
server_key: "0843f47d513874658fd600fc23aae749"
server_key_iv: "483e031719b62333f1cd1bad50c72bbf"
server_key_version: "encryptionkey@0"
I20260811 06:17:31.904808 1576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:31.933599 1576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:31.937774 1576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:31.938031 1576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:31.938910 1576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:31.940191 1576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:31.940274 1576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.940418 1576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:31.940478 1576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:31.971441 1576 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:45501
I20260811 06:17:31.971505 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:45501 every 8 connection(s)
I20260811 06:17:31.972658 1576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:31.976459 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 1576
I20260811 06:17:31.976655 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:31.977138 23012 external_mini_cluster.cc:1428] Setting key 2269de577b125e4fa5fc2ad60980cd63
Aug 11 06:17:31 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429051, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:31.991463 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:31.975108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:33529 (local address 127.22.121.62:33955)
0811 06:17:31.975347 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:31.975352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:31.976520 (+ 1168us) server_negotiation.cc:407] Connection header received
0811 06:17:31.978567 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:31.978572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:31.978660 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:31.978742 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:31.980300 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.981050 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.982615 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.983018 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.983124 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.986251 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:31.986256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:31.986259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:31.986290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:31.988653 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.989313 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.989316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.989319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.989446 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:31.989846 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:31.989850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:31.989852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:31.990106 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:31.990236 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:31.990875 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 06:17:31.991081 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:17:31.992344 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:31.975619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33955 (local address 127.22.121.3:33529)
0811 06:17:31.976375 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:31.976418 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:31.978293 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:31.978991 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:31.978999 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:31.979569 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:31.980127 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.980155 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.981202 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.981206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.982469 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:31.982481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:31.983235 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:31.983242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:31.983480 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:31.984036 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:31.984057 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:31.986051 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:31.988780 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.988788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.988803 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.989192 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.989551 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:31.989556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:31.989559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:31.989727 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:31.990229 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:31.990236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:31.990525 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 06:17:31.990741 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 06:17:31.991215 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":287,"threads_started":1}
I20260811 06:17:31.993598 1833 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33955
I20260811 06:17:31.993854 1833 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:31.994494 1833 heartbeater.cc:511] Master 127.22.121.62:33955 requested a full tablet report, sending...
I20260811 06:17:31.995827 942 ts_manager.cc:194] Registered new tserver with Master: 708b0db84b23404898be6c51a7c32708 (127.22.121.3:45501)
I20260811 06:17:31.996575 942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:33529
I20260811 06:17:32.003783 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:17:32.025763 1308 heartbeater.cc:503] Master 127.22.121.62:33955 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/test-admin.keytab.
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.032212 23012 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 06:17:32.036887 1846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:32.034495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.22.121.1:40563)
0811 06:17:32.035130 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.035138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.035163 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:17:32.035211 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.035218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.035474 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.035613 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.036520 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35214: BlockingRecv error: recv got EOF from 127.0.0.1:35214 (error 108)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":301,"threads_started":1}
W20260811 06:17:32.037051 23012 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.038564 23012 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.039773 23012 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:32.053782 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.041199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.22.121.1:40563)
0811 06:17:32.041365 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.041371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.041603 (+ 232us) server_negotiation.cc:407] Connection header received
0811 06:17:32.041758 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.041762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.041815 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.041905 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.042805 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.043784 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.044987 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.045517 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.045628 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.048858 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.048868 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.048881 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.048912 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.051538 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.052127 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.052133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.052136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.052270 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.052556 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.052559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.052561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.052873 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.053084 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.053348 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.053546 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 06:17:32.058918 1846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:32.057312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.22.121.1:40563)
0811 06:17:32.057458 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.057463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.057740 (+ 277us) server_negotiation.cc:407] Connection header received
0811 06:17:32.057872 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.057876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.057924 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.057996 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.058698 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35224: BlockingRecv error: recv got EOF from 127.0.0.1:35224 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 06:17:32.059212 23012 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.060623 23012 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.061750 23012 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:32.075241 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.062909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.22.121.1:40563)
0811 06:17:32.063082 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.063087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.063202 (+ 115us) server_negotiation.cc:407] Connection header received
0811 06:17:32.063360 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.063363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.063411 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.063480 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.064295 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.064932 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.066137 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.066740 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.066858 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.070541 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.070546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.070549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.070576 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.072880 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.073484 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.073489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.073492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.073610 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.073992 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.073997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.073999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.074246 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.074497 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.074783 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.074996 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 06:17:32.079196 1846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:32.077672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.22.121.1:40563)
0811 06:17:32.077803 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.077806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.078123 (+ 317us) server_negotiation.cc:407] Connection header received
0811 06:17:32.078242 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.078246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.078305 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.078373 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.079037 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35230: BlockingRecv error: recv got EOF from 127.0.0.1:35230 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 06:17:32.079497 23012 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429000386903-23012-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.080928 23012 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:32.082105 23012 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:32.094813 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.083218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.22.121.1:40563)
0811 06:17:32.083405 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.083411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.083673 (+ 262us) server_negotiation.cc:407] Connection header received
0811 06:17:32.083812 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.083817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.083876 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.083948 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.084829 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.085519 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.086692 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.087116 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.087222 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.090304 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.090309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.090312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.090340 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.092523 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.093080 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.093086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.093089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.093213 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.093571 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.093574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.093576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.093820 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.094122 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.094326 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.094515 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:17:32.096078 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 1060
I20260811 06:17:32.106441 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.096663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.22.121.1:40563)
0811 06:17:32.096836 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.096842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.097053 (+ 211us) server_negotiation.cc:407] Connection header received
0811 06:17:32.097149 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.097152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.097201 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.097272 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.098033 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.098657 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.099853 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.100239 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.100331 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.102072 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.102077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.102080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.102107 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.104411 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.104950 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.104954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.104956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.105088 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.105365 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.105368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.105370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.105598 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.105776 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.105981 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.106119 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:17:32.106951 1298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:32.188247 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 1060
I20260811 06:17:32.207957 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 1320
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:32.222530 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.208848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.22.121.2:34377)
0811 06:17:32.209405 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.209415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.209446 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:17:32.209509 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.209516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.209770 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.209896 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.210864 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.211990 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.213284 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.213825 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.213965 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.217589 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.217600 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.217614 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.217652 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.220208 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.220837 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.220843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.220846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.220951 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.221279 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.221282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.221284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.221594 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.221816 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.222069 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.222256 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":273,"threads_started":1}
I20260811 06:17:32.223801 1560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:32.303782 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 1320
I20260811 06:17:32.328466 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 1576
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:32.343375 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:32.329393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.22.121.3:45501)
0811 06:17:32.330023 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:32.330030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:32.330061 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:17:32.330143 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:32.330149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:32.330370 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:32.330497 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:32.331605 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.332681 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.333900 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:32.334452 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:32.334541 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:32.338330 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:32.338341 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:32.338356 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:32.338391 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:32.341081 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.341684 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.341690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.341693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.341804 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:32.342110 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:32.342113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:32.342115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:32.342418 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:32.342640 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:32.342844 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 06:17:32.343031 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":303,"threads_started":1}
I20260811 06:17:32.344596 1823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:32.424729 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 1576
I20260811 06:17:32.445719 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 896
I20260811 06:17:32.446892 997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:32.547854 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 896
2026-08-11T06:17:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3962 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[1882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[1882](info): set up 2 sockets
Aug 11 06:17:32 dist-test-slave-gjrz krb5kdc[1882](info): commencing operation
krb5kdc: starting...
W20260811 06:17:33.775384 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:33 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab.
Entry for principal oryx/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab.
I20260811 06:17:33.975464 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39221
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab
--principal=oryx/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:34.290494 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:34.290844 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:34.290968 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:34.298087 1915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:34.298180 1915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:34.298252 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:34.298327 1915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:34.298384 1915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:34.308166 1915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab
--principal=oryx/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:34.309372 1915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:34.310619 1915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:34.321616 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:34.321588 1931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:34.321806 1933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:34.322526 1915 server_base.cc:1034] running on GCE node
I20260811 06:17:34.323253 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:34.324563 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:34.325769 1915 hybrid_clock.cc:648] HybridClock initialized: now 1786429054325726 us; error 33 us; skew 500 ppm
I20260811 06:17:34.326061 1915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:34 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429054, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:34.328962 1915 init.cc:377] Logged in from keytab as oryx/127.22.121.62@KRBTEST.COM (short username oryx)
I20260811 06:17:34.330282 1915 webserver.cc:460] Webserver started at http://127.22.121.62:42077/ using document root <none> and password file <none>
I20260811 06:17:34.330891 1915 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:34.330986 1915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:34.331230 1915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:34.334059 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "477b028607914c79a70027d08838bbc2"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "850c819d354759d2c1810ce39159be03"
server_key_iv: "13b971ac11d17af384debd0588336663"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.334733 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "477b028607914c79a70027d08838bbc2"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "850c819d354759d2c1810ce39159be03"
server_key_iv: "13b971ac11d17af384debd0588336663"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.339416 1915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260811 06:17:34.342597 1944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:34.343822 1915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:34.343963 1915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "477b028607914c79a70027d08838bbc2"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "850c819d354759d2c1810ce39159be03"
server_key_iv: "13b971ac11d17af384debd0588336663"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.344152 1915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:34.360080 1915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:34.363961 1915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:34.364216 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:34.380359 1915 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:39221
I20260811 06:17:34.380388 2027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:39221 every 8 connection(s)
I20260811 06:17:34.381534 1915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:34.382836 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 1915
I20260811 06:17:34.383062 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:34.383942 23012 external_mini_cluster.cc:1428] Setting key af26abb71f6d73f8ebab26c9bb739429
I20260811 06:17:34.385757 2028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:17:34 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:34.398638 2028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Bootstrap starting.
I20260811 06:17:34.401619 2028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:34.402793 2028 log.cc:824] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:34.405691 2028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: No bootstrap required, opened a new log
I20260811 06:17:34.405900 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:34.385834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.22.121.62:39221)
0811 06:17:34.386628 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:34.386639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:34.386678 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:17:34.388226 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:34.388246 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:34.388602 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:34.388900 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:34.390617 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.392158 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.393800 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.394419 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.394563 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:34.399043 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:34.399052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:34.399065 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:34.399102 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:34.401950 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:34.402690 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:34.402695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:34.402701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:34.402854 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:34.403321 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:34.403326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:34.403329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:34.404038 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:34.404246 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:34.404664 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 06:17:34.404972 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":370,"threads_started":1}
I20260811 06:17:34.413241 2028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } }
I20260811 06:17:34.413549 2028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:34.413655 2028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477b028607914c79a70027d08838bbc2, State: Initialized, Role: FOLLOWER
I20260811 06:17:34.414204 2028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } }
I20260811 06:17:34.414420 2028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:34.414543 2028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:34.414695 2028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:34.416808 2028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } }
I20260811 06:17:34.417251 2028 leader_election.cc:302] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 477b028607914c79a70027d08838bbc2; no voters:
I20260811 06:17:34.417752 2028 leader_election.cc:288] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:34.417836 2035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:34.418476 2035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 LEADER]: Becoming Leader. State: Replica: 477b028607914c79a70027d08838bbc2, State: Running, Role: LEADER
I20260811 06:17:34.418990 2035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } }
I20260811 06:17:34.419585 2028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:34.421532 2036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } }
I20260811 06:17:34.421954 2036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:34.421522 2037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 477b028607914c79a70027d08838bbc2. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } }
I20260811 06:17:34.422348 2037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:34.423683 2042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:34.428437 2042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:34.437738 2042 catalog_manager.cc:1347] Generated new cluster ID: 64f9e51678b0499cab7d77115d58ec00
I20260811 06:17:34.437844 2042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:34.452081 2042 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:34.453812 2042 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:34.463712 2042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Generated new TSK 0
I20260811 06:17:34.464610 2042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
I20260811 06:17:34.510763 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:34.818974 2072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:34.819217 2072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:34.819368 2072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:34.826524 2072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:34.826627 2072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:34.826782 2072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:34.835980 2072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:34.837145 2072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:34.838413 2072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:34.850658 2082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:34.850706 2083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:34.850729 2086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:34.851508 2072 server_base.cc:1034] running on GCE node
I20260811 06:17:34.852046 2072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:34.852828 2072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:34.854014 2072 hybrid_clock.cc:648] HybridClock initialized: now 1786429054853965 us; error 43 us; skew 500 ppm
I20260811 06:17:34.854264 2072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:34 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429054, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:34.857148 2072 init.cc:377] Logged in from keytab as oryx/127.22.121.1@KRBTEST.COM (short username oryx)
I20260811 06:17:34.858428 2072 webserver.cc:460] Webserver started at http://127.22.121.1:38195/ using document root <none> and password file <none>
I20260811 06:17:34.859023 2072 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:34.859129 2072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:34.859439 2072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:34.862207 2072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "1dc41c93cfcd4986a926a68e7410f94a"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "9a164b150e9965bcb8ac600e1e6e77fd"
server_key_iv: "2a3574985b837266d37bb79658949906"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.862917 2072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "1dc41c93cfcd4986a926a68e7410f94a"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "9a164b150e9965bcb8ac600e1e6e77fd"
server_key_iv: "2a3574985b837266d37bb79658949906"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.867823 2072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:17:34.871047 2102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:34.872243 2072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:34.872380 2072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "1dc41c93cfcd4986a926a68e7410f94a"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "9a164b150e9965bcb8ac600e1e6e77fd"
server_key_iv: "2a3574985b837266d37bb79658949906"
server_key_version: "encryptionkey@0"
I20260811 06:17:34.872565 2072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:34.884307 2072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:34.887840 2072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:34.888170 2072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:34.889034 2072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:34.890245 2072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:34.890316 2072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:34.890424 2072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:34.890484 2072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:34.920039 2072 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33489
I20260811 06:17:34.920085 2311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33489 every 8 connection(s)
I20260811 06:17:34.921242 2072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:34.928052 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 2072
I20260811 06:17:34.928262 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:34.928875 23012 external_mini_cluster.cc:1428] Setting key b03c613f24b34f9692864a2434445dd7
Aug 11 06:17:34 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429054, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:34.941188 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:34.923663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:39377 (local address 127.22.121.62:39221)
0811 06:17:34.924057 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:34.924062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:34.925044 (+ 982us) server_negotiation.cc:407] Connection header received
0811 06:17:34.926842 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:34.926848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:34.926924 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:34.927022 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:34.928555 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.929294 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.931184 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.931646 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.931741 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:34.935481 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:34.935487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:34.935491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:34.935529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:34.938339 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:34.939001 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:34.939007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:34.939010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:34.939163 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:34.939662 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:34.939665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:34.939667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:34.939921 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:34.940030 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:34.940776 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 06:17:34.940942 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":201,"threads_started":1}
I20260811 06:17:34.942422 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:34.924202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:39377)
0811 06:17:34.924885 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:34.924923 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:34.926576 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:34.927259 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:34.927296 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:34.927792 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:34.928389 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:34.928409 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.929456 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.929460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:34.931036 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:34.931052 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:34.931868 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:34.931876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:34.932150 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:34.932824 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:34.932852 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:34.935236 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:34.938477 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:34.938484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:34.938495 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:34.938886 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:34.939303 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:34.939308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:34.939312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:34.939551 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:34.940038 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:34.940045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:34.940420 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 06:17:34.940708 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 06:17:34.941207 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":259,"threads_started":1}
I20260811 06:17:34.943831 2315 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:34.944123 2315 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:34.944783 2315 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
I20260811 06:17:34.946717 1967 ts_manager.cc:194] Registered new tserver with Master: 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489)
I20260811 06:17:34.948356 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:39377
WARNING: no policy specified for oryx/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
I20260811 06:17:34.977815 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:35.284795 2327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:35.285051 2327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:35.285161 2327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:35.292093 2327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:35.292191 2327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:35.292327 2327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:35.301515 2327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:35.302654 2327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:35.303895 2327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:35.316407 2336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:35.316368 2337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:35.316447 2340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:35.317386 2327 server_base.cc:1034] running on GCE node
I20260811 06:17:35.317972 2327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:35.318936 2327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:35.320137 2327 hybrid_clock.cc:648] HybridClock initialized: now 1786429055320088 us; error 40 us; skew 500 ppm
I20260811 06:17:35.320422 2327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:35 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429055, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:35.323400 2327 init.cc:377] Logged in from keytab as oryx/127.22.121.2@KRBTEST.COM (short username oryx)
I20260811 06:17:35.324774 2327 webserver.cc:460] Webserver started at http://127.22.121.2:41427/ using document root <none> and password file <none>
I20260811 06:17:35.325395 2327 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:35.325500 2327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:35.325815 2327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:35.328653 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "6c40232d035cd75063c2a8f29b5db706"
server_key_iv: "29db2108591e04ce6a3d116b0fa86182"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.329401 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "6c40232d035cd75063c2a8f29b5db706"
server_key_iv: "29db2108591e04ce6a3d116b0fa86182"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.334188 2327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:17:35.337422 2353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.338625 2327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:17:35.338757 2327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "6c40232d035cd75063c2a8f29b5db706"
server_key_iv: "29db2108591e04ce6a3d116b0fa86182"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.338925 2327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:35.363584 2327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:35.367255 2327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:35.367571 2327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:35.368474 2327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:35.369741 2327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:35.369817 2327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.369932 2327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:35.369982 2327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.401589 2327 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:42221
I20260811 06:17:35.401635 2568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:42221 every 8 connection(s)
I20260811 06:17:35.402870 2327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:35.404295 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 2327
I20260811 06:17:35.404510 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:35.405002 23012 external_mini_cluster.cc:1428] Setting key 466a09072976fd7a49e882d8b1779d2c
Aug 11 06:17:35 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429055, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:35.423673 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:35.405397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:51497 (local address 127.22.121.62:39221)
0811 06:17:35.405594 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:35.405600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:35.406915 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 06:17:35.409259 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:35.409263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:35.409324 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:35.409395 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:35.411196 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.411958 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.413991 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.414392 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.414487 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:35.418142 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:35.418147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:35.418151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:35.418182 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:35.420757 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:35.421575 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:35.421580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:35.421582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:35.421708 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:35.422155 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:35.422158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:35.422160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:35.422399 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:35.422514 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:35.423236 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 06:17:35.423431 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:17:35.424942 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:35.405933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:51497)
0811 06:17:35.406733 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:35.406789 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:35.408966 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:35.409634 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:35.409645 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:35.410286 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:35.410986 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:35.411008 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.412123 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.412128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:35.413864 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:35.413878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.414602 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.414609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:35.414860 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:35.415538 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:35.415564 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:35.417932 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:35.420884 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:35.420893 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:35.420909 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:35.421468 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:35.421813 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:35.421817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:35.421820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:35.422053 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:35.422530 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:35.422537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:35.422890 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 06:17:35.423179 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 06:17:35.423708 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":295,"threads_started":1}
I20260811 06:17:35.426294 2570 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:35.426569 2570 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:35.427222 2570 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.22.121.3@KRBTEST.COM; defaulting to no policy
I20260811 06:17:35.428660 1967 ts_manager.cc:194] Registered new tserver with Master: 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:35.429467 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:51497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
I20260811 06:17:35.455156 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:35.759742 2577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:35.759986 2577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:35.760083 2577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:35.766819 2577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:35.766906 2577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:35.767023 2577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:35.775964 2577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:35.777078 2577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:35.778293 2577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:35.790625 2585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:35.790664 2592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:35.790695 2586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:35.791546 2577 server_base.cc:1034] running on GCE node
I20260811 06:17:35.792094 2577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:35.792871 2577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:35.794066 2577 hybrid_clock.cc:648] HybridClock initialized: now 1786429055794021 us; error 37 us; skew 500 ppm
I20260811 06:17:35.794327 2577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:35 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429055, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:35.797252 2577 init.cc:377] Logged in from keytab as oryx/127.22.121.3@KRBTEST.COM (short username oryx)
I20260811 06:17:35.798549 2577 webserver.cc:460] Webserver started at http://127.22.121.3:45373/ using document root <none> and password file <none>
I20260811 06:17:35.799132 2577 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:35.799226 2577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:35.799521 2577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:35.802157 2577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "4622d1517748477bab51f2fa70971312"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "e11c8c18128e53cd5c08e9f4c55b3e77"
server_key_iv: "2b5931cccc671701a36fa5a3c7c099c1"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.802841 2577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "4622d1517748477bab51f2fa70971312"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "e11c8c18128e53cd5c08e9f4c55b3e77"
server_key_iv: "2b5931cccc671701a36fa5a3c7c099c1"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.807559 2577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260811 06:17:35.810628 2605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.811898 2577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:35.812040 2577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "4622d1517748477bab51f2fa70971312"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "e11c8c18128e53cd5c08e9f4c55b3e77"
server_key_iv: "2b5931cccc671701a36fa5a3c7c099c1"
server_key_version: "encryptionkey@0"
I20260811 06:17:35.812214 2577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:35.838172 2577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:35.842068 2577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:35.842329 2577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:35.843173 2577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:35.844410 2577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:35.844485 2577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.844599 2577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:35.844653 2577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:35.874567 2577 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42945
I20260811 06:17:35.874626 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42945 every 8 connection(s)
I20260811 06:17:35.875818 2577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:35.881126 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 2577
I20260811 06:17:35.881304 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:35.881726 23012 external_mini_cluster.cc:1428] Setting key cb36a63238a479e77622c3deef71145d
Aug 11 06:17:35 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429055, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:35.894404 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:35.878131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:42573 (local address 127.22.121.62:39221)
0811 06:17:35.878282 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:35.878286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:35.879428 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 06:17:35.881154 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:35.881160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:35.881244 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:35.881344 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:35.882881 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.883752 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.885360 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.885783 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.885890 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:35.889077 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:35.889081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:35.889085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:35.889118 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:35.891583 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:35.892241 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:35.892245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:35.892247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:35.892369 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:35.892782 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:35.892787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:35.892790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:35.893048 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:35.893178 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:35.893867 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 06:17:35.894080 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:17:35.895195 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:35.878566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:42573)
0811 06:17:35.879244 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:35.879319 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:35.880889 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:35.881554 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:35.881562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:35.882077 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:35.882715 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:35.882733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.883899 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.883902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:35.885217 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:35.885234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:35.885993 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:35.886001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:35.886224 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:35.886797 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:35.886818 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:35.888878 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:35.891713 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:35.891722 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:35.891735 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:35.892127 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:35.892475 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:35.892480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:35.892483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:35.892676 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:35.893175 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:35.893182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:35.893505 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 06:17:35.893724 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 06:17:35.894177 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":250,"threads_started":1}
I20260811 06:17:35.896503 2819 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:35.896763 2819 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:35.897361 2819 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
I20260811 06:17:35.898612 1967 ts_manager.cc:194] Registered new tserver with Master: 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945)
I20260811 06:17:35.899399 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:42573
I20260811 06:17:35.908342 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.121.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.61@KRBTEST.COM" created.
I20260811 06:17:35.950975 2315 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.61.keytab.
Entry for principal oryx/127.22.121.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.61.keytab.
I20260811 06:17:35.954493 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:39207
--webserver_interface=127.22.121.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.61.keytab
--principal=oryx/127.22.121.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39221,127.22.121.61:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:36.263131 2827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:36.263410 2827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:36.263541 2827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:36.270491 2827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:36.270576 2827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:36.270644 2827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:36.270714 2827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:36.270772 2827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:36.280001 2827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39221,127.22.121.61:39207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.61.keytab
--principal=oryx/127.22.121.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:39207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.121.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
W20260811 06:17:36.280288 2827 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.121.62:39221,127.22.121.61:39207'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 06:17:36.281185 2827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:36.282392 2827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:36.293290 2836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:36.293347 2837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:36.293258 2840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:36.294495 2827 server_base.cc:1034] running on GCE node
I20260811 06:17:36.295176 2827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:36.296437 2827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:36.297647 2827 hybrid_clock.cc:648] HybridClock initialized: now 1786429056297579 us; error 44 us; skew 500 ppm
I20260811 06:17:36.297958 2827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:36.301011 2827 init.cc:377] Logged in from keytab as oryx/127.22.121.61@KRBTEST.COM (short username oryx)
I20260811 06:17:36.302385 2827 webserver.cc:460] Webserver started at http://127.22.121.61:33753/ using document root <none> and password file <none>
I20260811 06:17:36.303025 2827 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:36.303122 2827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:36.303460 2827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:36.306246 2827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/instance:
uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "f5ab5049a682ce0e62b6841d2ceb54d7"
server_key_iv: "b039f4ff8fbd6f8f9c97ebfa75ce9f4b"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.306969 2827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal/instance:
uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "f5ab5049a682ce0e62b6841d2ceb54d7"
server_key_iv: "b039f4ff8fbd6f8f9c97ebfa75ce9f4b"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.311821 2827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 06:17:36.315021 2847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:36.316392 2827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:36.316568 2827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "f5ab5049a682ce0e62b6841d2ceb54d7"
server_key_iv: "b039f4ff8fbd6f8f9c97ebfa75ce9f4b"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.316764 2827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:36.339529 2827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:36.343376 2827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:36.343607 2827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:36.360323 2827 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.61:39207
I20260811 06:17:36.360330 2944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.61:39207 every 8 connection(s)
I20260811 06:17:36.361506 2827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
I20260811 06:17:36.365041 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:36.371021 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 2827
I20260811 06:17:36.371111 2945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.61@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
I20260811 06:17:36.391551 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.374086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46314 (local address 127.22.121.62:39221)
0811 06:17:36.374269 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.374274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.374741 (+ 467us) server_negotiation.cc:407] Connection header received
0811 06:17:36.375979 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.375983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.376053 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.376140 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.377544 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.378304 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.380033 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.380590 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.380728 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.384563 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.384569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.384574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.384610 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.388161 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.389249 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.389255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.389258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.389431 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.389868 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.389871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.389873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.390193 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.390327 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.391020 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.391237 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:17:36.392416 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.373919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46314)
0811 06:17:36.374573 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.374618 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.375720 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.376224 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.376230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.376971 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.377364 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.377385 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.378452 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.378458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.379876 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.379898 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.380855 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.380863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.381121 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.381774 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.381800 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.384349 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.388299 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.388310 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.388321 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.389136 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.389566 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.389570 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.389572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.389734 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.390327 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.390334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.390635 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.390876 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.391332 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"spinlock_wait_cycles":15616,"thread_start_us":266,"threads_started":1}
I20260811 06:17:36.392437 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.373078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.22.121.61:39207)
0811 06:17:36.373683 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.373693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.373729 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:17:36.375713 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.375740 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.376020 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.376203 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.377670 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.379059 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.380466 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.381125 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.381240 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.385551 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.385561 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.385576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.385617 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.389076 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.389825 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.389834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.389836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.389963 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.390330 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.390335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.390339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.390669 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.390980 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.391308 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.391532 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":264,"threads_started":1}
I20260811 06:17:36.394483 2945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.61@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
I20260811 06:17:36.406301 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.395211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60706)
0811 06:17:36.395422 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.395439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.395533 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.395803 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.395808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.396193 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.396321 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.396329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.397249 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.397254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.398055 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.398066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.398716 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.398721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.398884 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.399318 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.399338 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.401208 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.403988 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.403993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.403996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.404482 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.404824 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.404829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.404833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.404984 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.405465 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.405470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.405627 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.405775 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.406039 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:17:36.406353 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.395241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.22.121.61:39207)
0811 06:17:36.395406 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.395411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.395484 (+ 73us) server_negotiation.cc:407] Connection header received
0811 06:17:36.395628 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.395633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.395685 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.395763 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.396450 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.397105 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.398188 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.398582 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.398687 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.401432 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.401439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.401443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.401480 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.403853 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.404596 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.404603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.404605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.404732 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.405086 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.405089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.405091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.405335 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.405435 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.405851 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.406139 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":75}
I20260811 06:17:36.410475 2945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Bootstrap starting.
I20260811 06:17:36.413302 2945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:36.414234 2945 log.cc:824] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:36.416630 2945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: No bootstrap required, opened a new log
I20260811 06:17:36.423659 2945 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } }
I20260811 06:17:36.424016 2945 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:36.424106 2945 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3c75f4e3dbd477f8f713599130d3d8e, State: Initialized, Role: FOLLOWER
I20260811 06:17:36.424647 2945 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } }
I20260811 06:17:36.425472 2955 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } } }
I20260811 06:17:36.425762 2955 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:17:36.426358 2945 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:36.432029 2570 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
W20260811 06:17:36.434481 2978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:17:36.434594 2978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:17:36.438398 2827 master_runner.cc:428] Detected that this master c3c75f4e3dbd477f8f713599130d3d8e is joining an existing cluster
I20260811 06:17:36.438477 2827 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.121.61:39207
I20260811 06:17:36.458892 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.442023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.22.121.61:39207)
0811 06:17:36.442216 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.442222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.443508 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 06:17:36.443631 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.443640 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.443709 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.443819 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.445154 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.446064 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.448674 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.449364 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.449474 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.451998 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.452003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.452007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.452039 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.455841 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.456691 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.456696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.456699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.456852 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.457274 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.457278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.457281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.457629 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.457785 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.458309 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.458522 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":54}
I20260811 06:17:36.458894 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.441970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.22.121.62:39221)
0811 06:17:36.442134 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.442140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.443007 (+ 867us) server_negotiation.cc:407] Connection header received
0811 06:17:36.443333 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.443338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.443440 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.443510 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.445044 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.446166 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.448292 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.449002 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.449126 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.451742 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.451749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.451753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.451792 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.455045 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.456533 (+ 1488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.456539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.456542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.456693 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.457165 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.457170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.457173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.457536 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.457664 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.458321 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.458551 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:17:36.459961 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.442099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46322)
0811 06:17:36.442895 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.442916 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.443131 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.443705 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.443708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.444464 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.444876 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.444888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.446335 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.446340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.448122 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.448138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.449416 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.449424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.449614 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.450217 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.450241 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.451513 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.455194 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.455199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.455203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.456399 (+ 1196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.456848 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.456853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.456858 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.457033 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.457747 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.457752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.457960 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.459124 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.459445 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":24,"spinlock_wait_cycles":36736,"thread_start_us":506,"threads_started":1}
I20260811 06:17:36.459995 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.442590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60720)
0811 06:17:36.443416 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.443431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.443539 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.443867 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.443872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.444572 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.445028 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.445039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.446273 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.446278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.448515 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.448531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.449781 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.449787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.450267 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.450829 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.450854 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.451839 (+ 985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.455981 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.455986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.455989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.456561 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.457001 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.457005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.457008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.457161 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.457856 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.457860 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.458061 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.459444 (+ 1383us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.459672 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":1,"spinlock_wait_cycles":28544,"thread_start_us":517,"threads_started":1}
WARNING: no policy specified for oryx/127.22.121.60@KRBTEST.COM; defaulting to no policy
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429054, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.62@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.60@KRBTEST.COM" created.
I20260811 06:17:36.482828 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.466087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.22.121.61:39207)
0811 06:17:36.466233 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.466239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.466839 (+ 600us) server_negotiation.cc:407] Connection header received
0811 06:17:36.467191 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.467196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.467316 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.467414 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.468697 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.469714 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.471595 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.472281 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.472419 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.476107 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.476115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.476119 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.476154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.479575 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.480505 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.480511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.480514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.480669 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.481148 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.481153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.481157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.481488 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.481603 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.482270 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.482474 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:17:36.483912 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.466050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60736)
0811 06:17:36.466715 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.466763 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.466974 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.467496 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.467504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.468248 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.468505 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.468518 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.469859 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.469865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.471427 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.471442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.472458 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.472465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.472664 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.473296 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.473319 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.475862 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.479730 (+ 3868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.479741 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.479749 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.480367 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.480824 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.480830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.480833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.481010 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.481720 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.481728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.481946 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.483144 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.483495 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":298,"threads_started":1}
I20260811 06:17:36.486078 1967 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.121.61:39207: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
member_type: NON_VOTER
last_known_addr {
host: "127.22.121.61"
port: 39207
}
attrs {
promote: true
}
}
dest_uuid: "477b028607914c79a70027d08838bbc2"
cas_config_opid_index: -1
I20260811 06:17:36.487622 1967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } }
I20260811 06:17:36.498278 2894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:36.497923 2998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } }
I20260811 06:17:36.502029 2998 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.121.61:39207
I20260811 06:17:36.502211 2894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 06:17:36.503168 2827 master.cc:537] Master@127.22.121.61:39207 shutting down...
I20260811 06:17:36.502588 2998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } }
I20260811 06:17:36.504122 2998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:36.504199 2996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.002s
W20260811 06:17:36.509330 1950 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 06:17:36.510901 1950 consensus_peers.cc:487] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 -> Peer c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61:39207): Couldn't send request to peer c3c75f4e3dbd477f8f713599130d3d8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 06:17:36.514734 2827 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:36.515173 2827 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:36.515421 2827 tablet_replica.cc:330] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.60.keytab.
Entry for principal oryx/127.22.121.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.60.keytab.
I20260811 06:17:36.518389 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:39647
--webserver_interface=127.22.121.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.60.keytab
--principal=oryx/127.22.121.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:17:36.526434 2827 master.cc:552] Master@127.22.121.61:39207 shutdown complete.
I20260811 06:17:36.526573 2827 master_runner.cc:305] Clearing existing system tablet
I20260811 06:17:36.527040 2827 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:36.527143 2827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:36.529843 2827 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:36.531226 3009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:36.531527 2827 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:36.531769 2827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 06:17:36.531850 2827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "f5ab5049a682ce0e62b6841d2ceb54d7"
server_key_iv: "b039f4ff8fbd6f8f9c97ebfa75ce9f4b"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.535434 2827 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:36.540117 2827 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:17:36.540268 2827 log.cc:1196] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 06:17:36.540632 2827 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Deleting consensus metadata
I20260811 06:17:36.540853 2827 master_runner.cc:315] Copying system tablet from 127.22.121.62:39221
I20260811 06:17:36.548406 2827 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: tablet copy: Beginning tablet copy session from remote peer at address 127.22.121.62:39221
I20260811 06:17:36.560303 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.549299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46338 (local address 127.22.121.62:39221)
0811 06:17:36.549480 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:36.549487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:36.549931 (+ 444us) server_negotiation.cc:407] Connection header received
0811 06:17:36.550101 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:36.550107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:36.550187 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:36.550271 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:36.551116 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.551890 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.553092 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.553487 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.553599 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.555255 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:36.555282 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:36.555287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:36.555327 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:36.557816 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.558421 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.558425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.558427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.558550 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:36.558918 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:36.558922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:36.558924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:36.559195 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:36.559353 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:36.559762 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 06:17:36.559966 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:17:36.560338 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:36.549270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46338)
0811 06:17:36.549822 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:36.549841 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:36.549978 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:36.550321 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:36.550326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:36.550758 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:36.550967 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.550976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.552040 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.552046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.552936 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:36.552948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:36.553637 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:36.553643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:36.553814 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:36.554364 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:36.554381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:36.555076 (+ 695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:36.557967 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.557976 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.557980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.558303 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.558663 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:36.558668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:36.558671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:36.558809 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:36.559337 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:36.559341 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:36.559498 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 06:17:36.559628 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:17:36.559929 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":350,"threads_started":1}
I20260811 06:17:36.561403 2003 tablet_copy_service.cc:140] P 477b028607914c79a70027d08838bbc2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c3c75f4e3dbd477f8f713599130d3d8e ({username='oryx', principal='oryx/127.22.121.61@KRBTEST.COM'} at 127.0.0.1:46338)
I20260811 06:17:36.561730 2003 tablet_copy_service.cc:161] P 477b028607914c79a70027d08838bbc2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c3c75f4e3dbd477f8f713599130d3d8e at {username='oryx', principal='oryx/127.22.121.61@KRBTEST.COM'} at 127.0.0.1:46338: session id = c3c75f4e3dbd477f8f713599130d3d8e-00000000000000000000000000000000
I20260811 06:17:36.565017 2003 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:17:36.566987 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:17:36.571293 2827 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: tablet copy: Starting download of 0 data blocks...
I20260811 06:17:36.571623 2827 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: tablet copy: Starting download of 1 WAL segments...
I20260811 06:17:36.574379 2827 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:17:36.577682 2003 tablet_copy_service.cc:342] P 477b028607914c79a70027d08838bbc2: Request end of tablet copy session c3c75f4e3dbd477f8f713599130d3d8e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.121.61@KRBTEST.COM'} at 127.0.0.1:46338
I20260811 06:17:36.577832 2003 tablet_copy_service.cc:433] P 477b028607914c79a70027d08838bbc2: ending tablet copy session c3c75f4e3dbd477f8f713599130d3d8e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c3c75f4e3dbd477f8f713599130d3d8e
I20260811 06:17:36.588017 2827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:36.590338 3024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:36.595192 3028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:36.602497 2827 server_base.cc:1034] running on GCE node
W20260811 06:17:36.602972 3036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:36.603413 2827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:36.603995 2827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:36.605270 2827 hybrid_clock.cc:648] HybridClock initialized: now 1786429056605209 us; error 36 us; skew 500 ppm
I20260811 06:17:36.605473 2827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:36.607195 2827 init.cc:377] Logged in from keytab as oryx/127.22.121.61@KRBTEST.COM (short username oryx)
I20260811 06:17:36.608407 2827 webserver.cc:460] Webserver started at http://127.22.121.61:39371/ using document root <none> and password file <none>
I20260811 06:17:36.608762 2827 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:36.608837 2827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:36.611044 2827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 06:17:36.612656 3051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:36.613056 2827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 06:17:36.613185 2827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "f5ab5049a682ce0e62b6841d2ceb54d7"
server_key_iv: "b039f4ff8fbd6f8f9c97ebfa75ce9f4b"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.613322 2827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:36.633134 2827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:36.636236 2827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:36.661012 2827 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.61:39207
I20260811 06:17:36.661062 3132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.61:39207 every 8 connection(s)
I20260811 06:17:36.661863 2827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
I20260811 06:17:36.667439 3134 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:17:36.670483 3134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Bootstrap starting.
I20260811 06:17:36.684383 3134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:17:36.684814 3134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: Bootstrap complete.
I20260811 06:17:36.685601 3134 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } }
I20260811 06:17:36.685807 3134 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3c75f4e3dbd477f8f713599130d3d8e, State: Initialized, Role: LEARNER
I20260811 06:17:36.686062 3134 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } }
I20260811 06:17:36.686841 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } }
I20260811 06:17:36.687116 3134 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:36.687247 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: LEARNER
I20260811 06:17:36.695102 3161 catalog_manager.cc:1259] Loaded cluster ID: 64f9e51678b0499cab7d77115d58ec00
I20260811 06:17:36.695180 3161 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: loading cluster ID for follower catalog manager: success
I20260811 06:17:36.697433 3161 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: acquiring CA information for follower catalog manager: success
I20260811 06:17:36.699177 3161 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:17:36.901697 2819 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
W20260811 06:17:36.934845 3002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:36.935169 3002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:36.935376 3002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:36.943214 3002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:36.943336 3002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:36.943395 3002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:36.943511 3002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:36.943581 3002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:36.954789 3002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.60.keytab
--principal=oryx/127.22.121.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.121.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:36.955971 3002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:36.957280 3002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:36.971072 3169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:36.971356 3170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:36.971423 3174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:36.972537 3002 server_base.cc:1034] running on GCE node
I20260811 06:17:36.973261 3002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:36.974538 3002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:36.975766 3002 hybrid_clock.cc:648] HybridClock initialized: now 1786429056975706 us; error 35 us; skew 500 ppm
I20260811 06:17:36.976217 3002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:36 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:36.979955 3002 init.cc:377] Logged in from keytab as oryx/127.22.121.60@KRBTEST.COM (short username oryx)
I20260811 06:17:36.981387 3002 webserver.cc:460] Webserver started at http://127.22.121.60:33303/ using document root <none> and password file <none>
I20260811 06:17:36.982095 3002 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:36.982224 3002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:36.982558 3002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:36.985507 3002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/instance:
uuid: "08057ab7db9d4b34a88331bb3daddb30"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "3bddb536afb9176baa822c08193f6fb7"
server_key_iv: "d2fa6f95bc0d74a69e2536c5308113a3"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.986282 3002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal/instance:
uuid: "08057ab7db9d4b34a88331bb3daddb30"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "3bddb536afb9176baa822c08193f6fb7"
server_key_iv: "d2fa6f95bc0d74a69e2536c5308113a3"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.991375 3002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 06:17:36.994710 3184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:36.995996 3002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:36.996150 3002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "08057ab7db9d4b34a88331bb3daddb30"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "3bddb536afb9176baa822c08193f6fb7"
server_key_iv: "d2fa6f95bc0d74a69e2536c5308113a3"
server_key_version: "encryptionkey@0"
I20260811 06:17:36.996320 3002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:37.019409 3002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:37.023114 3002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:37.023402 3002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:37.049891 3002 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.60:39647
I20260811 06:17:37.050015 3280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.60:39647 every 8 connection(s)
I20260811 06:17:37.051393 3002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
I20260811 06:17:37.054394 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 3002
I20260811 06:17:37.056730 3283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:37.068011 3283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.60@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:37.085417 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.056998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.22.121.60:39647)
0811 06:17:37.058226 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.058238 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.058293 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:17:37.060744 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.060765 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.061121 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.061604 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.063761 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.065591 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.067168 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.068066 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.068245 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.074927 (+ 6682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.074936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.074945 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.075008 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.079510 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.080492 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.080501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.080508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.080792 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.081433 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.081438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.081442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.082163 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.083393 (+ 1230us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.083689 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.084075 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":648,"threads_started":1}
I20260811 06:17:37.089639 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.070645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46344)
0811 06:17:37.071760 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.071800 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.071984 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.072809 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.072816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.073790 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.074172 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.074183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.075433 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.075439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.077386 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.077402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.078759 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.078767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.078951 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.079569 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.079590 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.082510 (+ 2920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.086512 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.086522 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.086528 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.087089 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.087586 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.087591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.087594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.087758 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.088456 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.088465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.088685 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.088877 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.089204 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":614,"threads_started":1}
I20260811 06:17:37.091012 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.070829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.22.121.62:39221)
0811 06:17:37.072457 (+ 1628us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.072462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.072477 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:17:37.072526 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.072530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.072582 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.072677 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.074346 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.075228 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.077555 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.078513 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.078648 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.082748 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.082754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.082759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.082797 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.086371 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.087234 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.087239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.087242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.087403 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.087905 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.087910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.087913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.088290 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.090256 (+ 1966us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.090392 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.090599 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1526,"thread_start_us":1420,"threads_started":1}
I20260811 06:17:37.092360 3283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:17:37.097615 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.080054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60740)
0811 06:17:37.087454 (+ 7400us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.087472 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.087581 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.090902 (+ 3321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.090906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.090935 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:37.091393 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.091405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.093161 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.093167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.095642 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.095655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.097078 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.097083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.097210 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.097251 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.097305 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.097390 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7230,"thread_start_us":9950,"threads_started":1}
I20260811 06:17:37.097954 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.080252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.22.121.61:39207)
0811 06:17:37.090537 (+ 10285us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.090543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.090561 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:37.090641 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.090646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.090731 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.090852 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:37.091602 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.092917 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.095800 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.096883 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.097028 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.097374 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.097451 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.097588 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10148,"thread_start_us":10007,"threads_started":1}
I20260811 06:17:37.098894 3091 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 06:17:37.099474 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 477b028607914c79a70027d08838bbc2. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } }
I20260811 06:17:37.099741 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: LEARNER
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.60@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
I20260811 06:17:37.107174 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.093412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60754)
0811 06:17:37.093599 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.093616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.093690 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.095923 (+ 2233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.095927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.096203 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.096325 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.096332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.097993 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.097996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.099038 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.099060 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.100093 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.100100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.100240 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.100718 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.100734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.102598 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.105385 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.105388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.105390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.105742 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.106073 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.106076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.106078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.106189 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.106579 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.106582 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.106688 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.106782 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.106957 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:17:37.107290 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.093254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.22.121.61:39207)
0811 06:17:37.095631 (+ 2377us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.095635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.095650 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:17:37.095687 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.095690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.095749 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.095861 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.096464 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.097797 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.099209 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.099865 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.100053 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.102874 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.102881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.102884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.102914 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.105271 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.105825 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.105828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.105830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.105987 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.106263 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.106266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.106268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.106496 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.106600 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.106849 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.107017 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2271,"thread_start_us":2163,"threads_started":1}
I20260811 06:17:37.108170 3283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429056, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.60@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
I20260811 06:17:37.119673 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.108837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:45910)
0811 06:17:37.109040 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.109055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.109135 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.109352 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.109355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.109658 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.109798 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.109806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.110640 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.110643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.111624 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.111637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.112344 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.112349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.112619 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.113218 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.113234 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.115108 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.117662 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.117666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.117668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.118026 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.118328 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.118331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.118334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.118491 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.118918 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.118921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.119100 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.119212 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.119401 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:17:37.119949 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.108808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.22.121.60:39647)
0811 06:17:37.108932 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.108936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.109110 (+ 174us) server_negotiation.cc:407] Connection header received
0811 06:17:37.109206 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.109209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.109257 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.109327 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.109904 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.110540 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.111748 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.112158 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.112268 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.115257 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.115284 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.115289 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.115321 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.117548 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.118119 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.118122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.118124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.118234 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.118581 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.118584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.118586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.118827 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.118925 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.119435 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.119614 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:17:37.124619 3283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Bootstrap starting.
I20260811 06:17:37.127771 3283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:37.128973 3283 log.cc:824] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:37.131614 3283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: No bootstrap required, opened a new log
I20260811 06:17:37.139113 3283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } }
I20260811 06:17:37.139479 3283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:37.139578 3283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08057ab7db9d4b34a88331bb3daddb30, State: Initialized, Role: FOLLOWER
I20260811 06:17:37.140105 3283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } }
I20260811 06:17:37.141140 3296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } } }
I20260811 06:17:37.141616 3296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:17:37.142114 3283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:17:37.151676 3313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:17:37.151975 3313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:17:37.156049 3002 master_runner.cc:428] Detected that this master 08057ab7db9d4b34a88331bb3daddb30 is joining an existing cluster
I20260811 06:17:37.156204 3002 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.121.60:39647
I20260811 06:17:37.179826 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.160611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.22.121.60:39647)
0811 06:17:37.160912 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.160917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.162712 (+ 1795us) server_negotiation.cc:407] Connection header received
0811 06:17:37.162932 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.162937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.162997 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.163058 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.164666 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.165812 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.167946 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.169228 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.169335 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.172329 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.172334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.172339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.172370 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.176814 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.177580 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.177585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.177588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.177728 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.178177 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.178183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.178186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.178605 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.178725 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.179249 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.179465 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":52}
I20260811 06:17:37.180785 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.160330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:45912)
0811 06:17:37.162615 (+ 2285us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.162639 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.162804 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.163256 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.163283 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.164131 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.164500 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.164512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.166005 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.166009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.167807 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.167822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.169768 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.169775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.169980 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.170698 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.170725 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.172135 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.176925 (+ 4790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.176929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.176933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.177462 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.177861 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.177866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.177869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.178043 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.178794 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.178798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.178967 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.179989 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.180299 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2144,"thread_start_us":1365,"threads_started":1}
I20260811 06:17:37.181816 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.160320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.22.121.61:39207)
0811 06:17:37.160484 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.160490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.162779 (+ 2289us) server_negotiation.cc:407] Connection header received
0811 06:17:37.162980 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.162984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.163052 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.163135 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.165806 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.167029 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.169297 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.169928 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.170022 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.172532 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.172538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.172543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.172594 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.178499 (+ 5905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.179461 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.179466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.179469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.179626 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.180071 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.180075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.180079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.180407 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.180546 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.181273 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.181513 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 06:17:37.182823 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.161485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60762)
0811 06:17:37.162700 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.162714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.162851 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.164322 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.164327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.165064 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.165643 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.165657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.167232 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.167237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.169120 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.169136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.170311 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.170318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.170484 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.171198 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.171224 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.172332 (+ 1108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.178845 (+ 6513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.178849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.178853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.179334 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.179768 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.179772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.179776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.179942 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.180730 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.180735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.180959 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.182065 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.182353 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":20,"thread_start_us":823,"threads_started":1}
I20260811 06:17:37.185174 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.160196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.22.121.62:39221)
0811 06:17:37.160375 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.160381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.169111 (+ 8730us) server_negotiation.cc:407] Connection header received
0811 06:17:37.169588 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.169594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.169672 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.169740 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.171924 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.173169 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.175448 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.176186 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.176320 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.178751 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.178758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.178762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.178804 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.182341 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.183166 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.183171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.183174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.183365 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.183729 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.183734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.183738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.184007 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.184158 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.184650 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.184852 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:17:37.185231 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.162065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46346)
0811 06:17:37.169011 (+ 6946us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.169029 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.169421 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.169939 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.169944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.170802 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.171743 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.171758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.173382 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.173387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.175304 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.175319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.176402 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.176449 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.176630 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.177328 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.177352 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.178513 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.182523 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.182528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.182531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.183046 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.183468 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.183473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.183477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.183615 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.184219 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.184222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.184383 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.184503 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.184767 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6767,"thread_start_us":7069,"threads_started":1}
I20260811 06:17:37.192683 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 2072
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429054, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.62@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.1@KRBTEST.COM
I20260811 06:17:37.207352 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.190662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:45918)
0811 06:17:37.190830 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.190848 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.190931 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.191408 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.191413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.191849 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.192040 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.192051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.193604 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.193610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.195161 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.195178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.196353 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.196358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.196583 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.197390 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.197411 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.200370 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.204843 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.204848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.204852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.205369 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.205795 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.205800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.205803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.205955 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.206538 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.206543 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.206682 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.206801 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.207053 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 06:17:37.208104 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.190803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.22.121.60:39647)
0811 06:17:37.191000 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.191006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.191024 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:37.191106 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.191110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.191198 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.191350 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.192200 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.193429 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.195376 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.196182 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.196316 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.200583 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.200589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.200594 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.200630 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.204707 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.205503 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.205508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.205512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.205659 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.206092 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.206097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.206100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.206401 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.206518 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.207463 (+ 945us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.207699 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:17:37.209265 1967 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.121.60:39647: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30"
member_type: NON_VOTER
last_known_addr {
host: "127.22.121.60"
port: 39647
}
attrs {
promote: true
}
}
dest_uuid: "477b028607914c79a70027d08838bbc2"
cas_config_opid_index: 5
I20260811 06:17:37.210098 1967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } }
I20260811 06:17:37.211809 3289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.212440 3091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 06:17:37.215749 3329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 06:17:37.216316 3233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:37.218075 3091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.221065 3233 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 06:17:37.221900 3289 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.121.60:39647
I20260811 06:17:37.222298 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.222595 3002 master.cc:537] Master@127.22.121.60:39647 shutting down...
I20260811 06:17:37.222641 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: LEARNER
I20260811 06:17:37.223320 3289 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:37.223557 3329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: NON_VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: true } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.224117 3329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:37.226676 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.193628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.22.121.1:33489)
0811 06:17:37.194295 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.194301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.194327 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:17:37.194381 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.194387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.194586 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.194724 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.196528 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.198416 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.200343 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.201055 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.201171 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.206006 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.206019 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.206037 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.206076 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.209995 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.210923 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.210933 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.210936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.211141 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.224000 (+ 12859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.224006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.224009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.224476 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.224816 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.225949 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.226250 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":350,"threads_started":1}
I20260811 06:17:37.228463 2294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:37.233525 3329 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: attempting to promote NON_VOTER c3c75f4e3dbd477f8f713599130d3d8e to VOTER
I20260811 06:17:37.234601 3329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } }
I20260811 06:17:37.235616 3002 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:37.236021 3002 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:37.236196 3002 tablet_replica.cc:330] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: stopping tablet replica
I20260811 06:17:37.237052 3091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 LEARNER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:17:37.237691 3289 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 06:17:37.241052 3334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.241706 3091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.245515 3002 master.cc:552] Master@127.22.121.60:39647 shutdown complete.
I20260811 06:17:37.245615 3002 master_runner.cc:305] Clearing existing system tablet
I20260811 06:17:37.245397 3334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.245601 3289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.245899 3289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:37.245939 3334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:37.246045 3002 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:37.246161 3002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:37.246311 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.246496 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 06:17:37.247073 1946 connection.cc:698] client connection to 127.22.121.60:39647 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260811 06:17:37.248452 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.247977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:45926)
0811 06:17:37.248138 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.248317 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:39647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:17:37.248745 1946 consensus_peers.cc:487] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 -> Peer 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60:39647): Couldn't send request to peer 08057ab7db9d4b34a88331bb3daddb30. Status: Network error: Client connection negotiation failed: client connection to 127.22.121.60:39647: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:17:37.253234 3002 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:17:37.254909 3341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:37.255297 3002 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:37.255573 3002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 06:17:37.255679 3002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "08057ab7db9d4b34a88331bb3daddb30"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "3bddb536afb9176baa822c08193f6fb7"
server_key_iv: "d2fa6f95bc0d74a69e2536c5308113a3"
server_key_version: "encryptionkey@0"
I20260811 06:17:37.260397 3002 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:37.265760 3002 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:17:37.265918 3002 log.cc:1196] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 06:17:37.266422 3002 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Deleting consensus metadata
I20260811 06:17:37.266651 3002 master_runner.cc:315] Copying system tablet from 127.22.121.62:39221
I20260811 06:17:37.269157 3002 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: tablet copy: Beginning tablet copy session from remote peer at address 127.22.121.62:39221
I20260811 06:17:37.281291 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.269840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.22.121.62:39221)
0811 06:17:37.270138 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.270144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.270604 (+ 460us) server_negotiation.cc:407] Connection header received
0811 06:17:37.270819 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.270825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.270905 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.270989 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.271944 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.272628 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.274002 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.274427 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.274539 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.276269 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.276275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.276280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.276311 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.278794 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.279398 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.279403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.279407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.279536 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.279924 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.279928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.279930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.280189 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.280326 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.280747 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.280961 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260811 06:17:37.281354 3355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.269791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:46352)
0811 06:17:37.270490 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.270508 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.270676 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.271042 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.271047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.271557 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.271774 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.271784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.272781 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.272787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.273853 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.273865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.274571 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.274577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.274770 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.275359 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.275377 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.276081 (+ 704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.278931 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.278936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.278941 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.279299 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.279648 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.279653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.279657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.279798 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.280316 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.280320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.280494 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.280603 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.280872 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":420,"threads_started":1}
I20260811 06:17:37.281932 2003 tablet_copy_service.cc:140] P 477b028607914c79a70027d08838bbc2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 08057ab7db9d4b34a88331bb3daddb30 ({username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:46352)
I20260811 06:17:37.282080 2003 tablet_copy_service.cc:161] P 477b028607914c79a70027d08838bbc2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 08057ab7db9d4b34a88331bb3daddb30 at {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:46352: session id = 08057ab7db9d4b34a88331bb3daddb30-00000000000000000000000000000000
I20260811 06:17:37.284566 2003 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:17:37.286178 3002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:17:37.291554 3002 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: tablet copy: Starting download of 0 data blocks...
I20260811 06:17:37.291795 3002 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: tablet copy: Starting download of 1 WAL segments...
I20260811 06:17:37.294212 3002 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:17:37.298105 2003 tablet_copy_service.cc:342] P 477b028607914c79a70027d08838bbc2: Request end of tablet copy session 08057ab7db9d4b34a88331bb3daddb30-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:46352
I20260811 06:17:37.298254 2003 tablet_copy_service.cc:433] P 477b028607914c79a70027d08838bbc2: ending tablet copy session 08057ab7db9d4b34a88331bb3daddb30-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 08057ab7db9d4b34a88331bb3daddb30
I20260811 06:17:37.309895 3002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:37.312608 3356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:37.313366 3357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:37.325604 3002 server_base.cc:1034] running on GCE node
W20260811 06:17:37.327529 3360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:37.328024 3002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:37.328606 3002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:37.329806 3002 hybrid_clock.cc:648] HybridClock initialized: now 1786429057329777 us; error 28 us; skew 500 ppm
I20260811 06:17:37.330024 3002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:37.331946 3002 init.cc:377] Logged in from keytab as oryx/127.22.121.60@KRBTEST.COM (short username oryx)
I20260811 06:17:37.336333 3002 webserver.cc:460] Webserver started at http://127.22.121.60:46861/ using document root <none> and password file <none>
I20260811 06:17:37.336611 3002 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:37.336735 3002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:37.339289 3002 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:17:37.341187 3371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:37.341665 3002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 06:17:37.341816 3002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "08057ab7db9d4b34a88331bb3daddb30"
format_stamp: "Formatted at 2026-08-11 06:17:36 on dist-test-slave-gjrz"
server_key: "3bddb536afb9176baa822c08193f6fb7"
server_key_iv: "d2fa6f95bc0d74a69e2536c5308113a3"
server_key_version: "encryptionkey@0"
I20260811 06:17:37.342085 3002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:37.365533 3002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:37.369063 3002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:37.379200 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 2072
I20260811 06:17:37.419088 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:33489
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:39221
--webserver_port=38195
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:17:37.432686 3002 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.60:39647
I20260811 06:17:37.432740 3464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.60:39647 every 8 connection(s)
I20260811 06:17:37.433705 3002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
I20260811 06:17:37.444272 3466 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:17:37.448230 3466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Bootstrap starting.
I20260811 06:17:37.466245 3466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:17:37.466727 3466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Bootstrap complete.
I20260811 06:17:37.467753 3466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } }
I20260811 06:17:37.468040 3466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 08057ab7db9d4b34a88331bb3daddb30, State: Initialized, Role: LEARNER
I20260811 06:17:37.468297 3466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } }
I20260811 06:17:37.469033 3472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.469357 3466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:37.469601 3472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:17:37.479492 3489 catalog_manager.cc:1259] Loaded cluster ID: 64f9e51678b0499cab7d77115d58ec00
I20260811 06:17:37.479575 3489 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: loading cluster ID for follower catalog manager: success
I20260811 06:17:37.482933 3489 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: acquiring CA information for follower catalog manager: success
I20260811 06:17:37.485210 3489 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:17:37.768168 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.760519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:45936)
0811 06:17:37.761408 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.761430 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.761601 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.761974 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.761980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.762009 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:37.762258 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.762272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.763856 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.763862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.765815 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.765832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.767370 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.767376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.767565 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.767602 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.767709 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.767846 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":659,"threads_started":1}
I20260811 06:17:37.768297 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.760695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.22.121.60:39647)
0811 06:17:37.761262 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.761268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.761527 (+ 259us) server_negotiation.cc:407] Connection header received
0811 06:17:37.761732 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.761736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.761814 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.761925 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:37.762393 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.763697 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.765979 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.767146 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.767301 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.767495 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.767735 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.767858 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":262,"threads_started":1}
I20260811 06:17:37.769698 3411 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 06:17:37.770452 3472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 477b028607914c79a70027d08838bbc2. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:37.770830 3472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: LEARNER
W20260811 06:17:37.792228 3443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:37.792484 3443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:37.792621 3443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:37.800406 3443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:37.800516 3443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:37.800704 3443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:37.810782 3443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:33489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=38195
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:37.811904 3443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:37.813196 3443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:37.827055 3508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:37.827608 3514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:37.828083 3518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:37.829185 3443 server_base.cc:1034] running on GCE node
I20260811 06:17:37.829782 3443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:37.830680 3443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:37.831878 3443 hybrid_clock.cc:648] HybridClock initialized: now 1786429057831820 us; error 45 us; skew 500 ppm
I20260811 06:17:37.832222 3443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:37.835233 3443 init.cc:377] Logged in from keytab as oryx/127.22.121.1@KRBTEST.COM (short username oryx)
I20260811 06:17:37.836768 3443 webserver.cc:460] Webserver started at http://127.22.121.1:38195/ using document root <none> and password file <none>
I20260811 06:17:37.837383 3443 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:37.837492 3443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:37.842806 3443 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:17:37.845516 3529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:37.846823 3443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:17:37.846971 3443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "1dc41c93cfcd4986a926a68e7410f94a"
format_stamp: "Formatted at 2026-08-11 06:17:34 on dist-test-slave-gjrz"
server_key: "9a164b150e9965bcb8ac600e1e6e77fd"
server_key_iv: "2a3574985b837266d37bb79658949906"
server_key_version: "encryptionkey@0"
I20260811 06:17:37.847990 3443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:37.866815 3443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:37.871121 3443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:37.871420 3443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:37.872536 3443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:37.874519 3443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:37.874697 3443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:37.874843 3443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:37.874922 3443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:37.921419 3443 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33489
I20260811 06:17:37.921485 3746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33489 every 8 connection(s)
I20260811 06:17:37.922807 3443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:37.927841 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 3443
I20260811 06:17:37.928190 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 2327
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
Aug 11 06:17:37 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.2@KRBTEST.COM
I20260811 06:17:37.958475 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.925679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42655 (local address 127.22.121.60:39647)
0811 06:17:37.925841 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.925846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.936704 (+ 10858us) server_negotiation.cc:407] Connection header received
0811 06:17:37.937789 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.937796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.937894 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.937985 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.940873 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942040 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.945130 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.945732 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.945832 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.951649 (+ 5817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.951657 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.951661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.951702 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.955290 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.956328 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.956332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.956335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.956473 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.956907 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.956912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.956914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.957243 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.957365 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.957888 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.958107 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 06:17:37.959551 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.926101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.22.121.1:42655)
0811 06:17:37.936593 (+ 10492us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.936612 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.937466 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.938425 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.938435 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.939497 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.940702 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.940730 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942998 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.943003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.944969 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.944984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.946289 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.946297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.946597 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.947436 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.947474 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.951383 (+ 3909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.955446 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.955451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.955455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.956192 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.956628 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.956633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.956636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.956783 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.957441 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.957446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.957602 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.958727 (+ 1125us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.958974 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10315,"mutex_wait_us":603,"thread_start_us":9742,"threads_started":1}
I20260811 06:17:37.959679 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.926101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:38591)
0811 06:17:37.932828 (+ 6727us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.932885 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.937463 (+ 4578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.939069 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.939073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.939767 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.940854 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.940863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942306 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.942310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.944794 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.944812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.946550 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.946559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.946803 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.947690 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.947714 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.951982 (+ 4268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.957062 (+ 5080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.957066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.957069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.957502 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.957914 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.957919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.957921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.958107 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.958750 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.958755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.958910 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.959034 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.959318 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6409,"mutex_wait_us":527,"thread_start_us":6160,"threads_started":1}
I20260811 06:17:37.960171 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.925387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:37837 (local address 127.22.121.61:39207)
0811 06:17:37.938392 (+ 13005us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.938398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.938450 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:17:37.938509 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.938514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.938631 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.938733 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.940897 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942134 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.944966 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.945773 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.945890 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.951413 (+ 5523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.951429 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.951434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.951475 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.954810 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.955779 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.955785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.955788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.955934 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.956461 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.956466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.956470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.956797 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.956930 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.959577 (+ 2647us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.959807 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12883,"thread_start_us":12777,"threads_started":1}
I20260811 06:17:37.960490 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.927186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.22.121.1:37837)
0811 06:17:37.937190 (+ 10004us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:37.937207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:37.937830 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:37.938907 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:37.938911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:37.939767 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:37.940695 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.940722 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942309 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.942314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.944794 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:37.944812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.946249 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.946261 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:37.946651 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.947442 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:37.947523 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:37.951166 (+ 3643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:37.954979 (+ 3813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.954987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.955002 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.955630 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.956070 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:37.956074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:37.956078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:37.956304 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:37.957004 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:37.957011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:37.957377 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 06:17:37.957706 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 06:17:37.958177 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9858,"mutex_wait_us":369,"spinlock_wait_cycles":19328,"thread_start_us":9576,"threads_started":1}
I20260811 06:17:37.960767 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.926341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:38591 (local address 127.22.121.62:39221)
0811 06:17:37.926939 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.926946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.935742 (+ 8796us) server_negotiation.cc:407] Connection header received
0811 06:17:37.938732 (+ 2990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.938738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.938809 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.938904 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.941009 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942149 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.944966 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.945782 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.945889 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.953321 (+ 7432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.953327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.953332 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.953370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.956929 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.957634 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.957639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.957642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.957794 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.958235 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.958240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.958244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.958561 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.958692 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.960219 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.960419 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":327,"threads_started":1}
I20260811 06:17:37.961351 3751 heartbeater.cc:347] Connected to a master server at 127.22.121.61:39207
I20260811 06:17:37.961484 3748 heartbeater.cc:347] Connected to a master server at 127.22.121.60:39647
I20260811 06:17:37.961668 3748 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:37.961671 3751 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:37.961938 3750 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:37.962083 3750 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:37.962858 3751 heartbeater.cc:511] Master 127.22.121.61:39207 requested a full tablet report, sending...
I20260811 06:17:37.962885 3750 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
I20260811 06:17:37.962909 3748 heartbeater.cc:511] Master 127.22.121.60:39647 requested a full tablet report, sending...
I20260811 06:17:37.963640 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:37.929628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.22.121.2:42221)
0811 06:17:37.937945 (+ 8317us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:37.937953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:37.937982 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:17:37.938062 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:37.938071 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:37.938298 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:37.938466 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:37.940771 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.942411 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.944410 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:37.945270 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:37.945371 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:37.953503 (+ 8132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:37.953517 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:37.953532 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:37.953571 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:37.957633 (+ 4062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.958899 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.958907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.958910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.959168 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:37.960135 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:37.960140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:37.960143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:37.961132 (+ 989us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:37.961365 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:37.962271 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 06:17:37.962515 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8125,"thread_start_us":7927,"threads_started":1}
I20260811 06:17:37.964917 1967 ts_manager.cc:194] Re-registered known tserver with Master: 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489)
I20260811 06:17:37.965090 2558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:37.965258 3076 ts_manager.cc:194] Registered new tserver with Master: 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489)
I20260811 06:17:37.965282 3397 ts_manager.cc:194] Registered new tserver with Master: 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489)
I20260811 06:17:37.965950 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:38591
I20260811 06:17:38.048426 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 2327
I20260811 06:17:38.077589 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:42221
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:39221
--webserver_port=41427
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:17:38.282374 3490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: NON_VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: true } } }
I20260811 06:17:38.282737 3490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:38.284389 3774 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: attempting to promote NON_VOTER 08057ab7db9d4b34a88331bb3daddb30 to VOTER
I20260811 06:17:38.285005 3774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } }
I20260811 06:17:38.287079 3091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:17:38.287138 3411 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 LEARNER]: Refusing update from remote peer 477b028607914c79a70027d08838bbc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:17:38.287782 3775 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:17:38.288275 3490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:17:38.292323 3490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.296221 3490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.296475 3490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:38.296756 3411 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.297355 3490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.297576 3490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:38.297983 3091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.306238 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.306418 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:38.305647 3789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.307129 3789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:17:38.307657 3787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477b028607914c79a70027d08838bbc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:38.307982 3787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 06:17:38.399144 3773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:38.399392 3773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:38.399523 3773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:38.406378 3773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:38.406471 3773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:38.406618 3773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:38.415803 3773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:42221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=41427
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:38.416953 3773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:38.418162 3773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:38.430473 3798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:38.430521 3799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:38.430549 3801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:38.431325 3773 server_base.cc:1034] running on GCE node
I20260811 06:17:38.431897 3773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:38.432790 3773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:38.433979 3773 hybrid_clock.cc:648] HybridClock initialized: now 1786429058433926 us; error 44 us; skew 500 ppm
I20260811 06:17:38.434325 3773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:38 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:38.437508 3773 init.cc:377] Logged in from keytab as oryx/127.22.121.2@KRBTEST.COM (short username oryx)
I20260811 06:17:38.440518 3773 webserver.cc:460] Webserver started at http://127.22.121.2:41427/ using document root <none> and password file <none>
I20260811 06:17:38.441129 3773 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:38.441255 3773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:38.446385 3773 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260811 06:17:38.448961 3813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:38.450147 3773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:38.450294 3773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "6c40232d035cd75063c2a8f29b5db706"
server_key_iv: "29db2108591e04ce6a3d116b0fa86182"
server_key_version: "encryptionkey@0"
I20260811 06:17:38.451207 3773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:38.464572 3773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:38.468050 3773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:38.468310 3773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:38.469159 3773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:38.470474 3773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:38.470564 3773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:38.470681 3773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:38.470754 3773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:38.501322 3773 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:42221
I20260811 06:17:38.501386 4018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:42221 every 8 connection(s)
I20260811 06:17:38.502597 3773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:38.508371 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 3773
I20260811 06:17:38.508662 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 2577
Aug 11 06:17:38 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
Aug 11 06:17:38 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.3@KRBTEST.COM
Aug 11 06:17:38 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
Aug 11 06:17:38 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
I20260811 06:17:38.555915 4026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.505106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49137 (local address 127.22.121.62:39221)
0811 06:17:38.534531 (+ 29425us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:38.534538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:38.534558 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:17:38.534636 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:38.534641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:38.534756 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:38.537229 (+ 2473us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:38.537391 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.538658 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.541302 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542244 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.542376 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.547878 (+ 5502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:38.547883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:38.547886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:38.547926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:38.551582 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.552405 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.552409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.552411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.552582 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.553202 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.553207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.553210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.553546 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:38.554911 (+ 1365us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:38.555180 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:17:38.555427 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29294,"thread_start_us":34983,"threads_started":1}
I20260811 06:17:38.557503 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.505126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:60099 (local address 127.22.121.61:39207)
0811 06:17:38.524186 (+ 19060us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:38.524192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:38.539129 (+ 14937us) server_negotiation.cc:407] Connection header received
0811 06:17:38.539245 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:38.539250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:38.539363 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:38.539471 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:38.541099 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542261 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.544446 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.545069 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.545178 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.549722 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:38.549729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:38.549733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:38.549786 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:38.554372 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555189 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.555194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.555197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.555365 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555832 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.555837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.555840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.556168 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:38.556306 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:38.556957 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 06:17:38.557164 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18934,"thread_start_us":19060,"threads_started":1}
I20260811 06:17:38.557713 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.505701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:49137)
0811 06:17:38.516137 (+ 10436us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:38.516187 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:38.521775 (+ 5588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:38.535291 (+ 13516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:38.535304 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:38.536188 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:38.537032 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.537056 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.538854 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.538860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.541117 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.541129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542683 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.542692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.543066 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.543979 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:38.544011 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:38.547624 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:38.551700 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.551709 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.551725 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.552266 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.552801 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.552806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.552810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.553060 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.553743 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:38.553752 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:38.554132 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 06:17:38.554423 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 06:17:38.556324 (+ 1901us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10133,"thread_start_us":32906,"threads_started":1}
I20260811 06:17:38.558315 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.505454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:59695 (local address 127.22.121.60:39647)
0811 06:17:38.533773 (+ 28319us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:38.533780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:38.538947 (+ 5167us) server_negotiation.cc:407] Connection header received
0811 06:17:38.539016 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:38.539021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:38.539121 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:38.539226 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:38.541506 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542615 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.544752 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.545601 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.545726 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.550394 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:38.550400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:38.550405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:38.550441 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:38.554061 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.554991 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.554996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.554999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.555155 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555601 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.555605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.555608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.555931 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:38.556304 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:38.557426 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0811 06:17:38.557929 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28189,"thread_start_us":28337,"threads_started":1}
I20260811 06:17:38.558806 4030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.506292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.22.121.2:60099)
0811 06:17:38.539007 (+ 32715us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:38.539023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:38.539139 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:38.539633 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:38.539638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:38.540503 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:38.540894 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.540907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542456 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.542463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.544271 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.544289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.545440 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.545450 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.545621 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.546244 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:38.546265 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:38.549484 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:38.554522 (+ 5038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.554527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.554530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.555057 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.555496 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.555504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.555702 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.556382 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:38.556387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:38.556526 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 06:17:38.556694 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 06:17:38.558555 (+ 1861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32578,"mutex_wait_us":1641,"thread_start_us":9426,"threads_started":1}
I20260811 06:17:38.558877 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.510040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.22.121.3:42945)
0811 06:17:38.532221 (+ 22181us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:38.532229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:38.532898 (+ 669us) server_negotiation.cc:407] Connection header received
0811 06:17:38.532988 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:38.532997 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:38.533297 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:38.533454 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:38.536234 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.538067 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.540981 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542009 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.542132 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.550224 (+ 8092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:38.550238 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:38.550254 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:38.550303 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:38.554372 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555213 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.555221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.555316 (+ 95us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.555479 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555934 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:38.555939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:38.555942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:38.556430 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:38.556698 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:38.557773 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0811 06:17:38.558394 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22006,"thread_start_us":21846,"threads_started":1}
I20260811 06:17:38.559062 4033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:38.505701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.22.121.2:59695)
0811 06:17:38.538669 (+ 32968us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:38.538690 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:38.538820 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:38.539461 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:38.539466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:38.540638 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:38.541316 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.541333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.542794 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.542799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.544567 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:38.544583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:38.546031 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:38.546039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:38.546301 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:38.547888 (+ 1587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:38.547908 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:38.550183 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:38.554218 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.554222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.554225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.554851 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.555314 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:38.555319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:38.555322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:38.555462 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:38.556365 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:38.556369 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:38.556512 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 06:17:38.556644 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 06:17:38.556919 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32815,"mutex_wait_us":30,"spinlock_wait_cycles":26368,"thread_start_us":32489,"threads_started":1}
I20260811 06:17:38.560030 4019 heartbeater.cc:347] Connected to a master server at 127.22.121.60:39647
I20260811 06:17:38.560062 4022 heartbeater.cc:347] Connected to a master server at 127.22.121.61:39207
I20260811 06:17:38.560304 2810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:38.560320 4019 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:38.560326 4022 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:38.561302 4019 heartbeater.cc:511] Master 127.22.121.60:39647 requested a full tablet report, sending...
I20260811 06:17:38.561347 4022 heartbeater.cc:511] Master 127.22.121.61:39207 requested a full tablet report, sending...
I20260811 06:17:38.562523 4020 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:38.562677 4020 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:38.563191 4020 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
I20260811 06:17:38.563210 3076 ts_manager.cc:194] Registered new tserver with Master: 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:38.563808 3397 ts_manager.cc:194] Registered new tserver with Master: 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:38.564975 1967 ts_manager.cc:194] Re-registered known tserver with Master: 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:38.566191 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:49137
I20260811 06:17:38.644575 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 2577
I20260811 06:17:38.669826 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:42945
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:39221
--webserver_port=45373
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:17:38.968204 3750 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
W20260811 06:17:38.989107 4043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:38.989349 4043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:38.989488 4043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:38.996466 4043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:38.996557 4043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:38.996698 4043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:39.005822 4043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:42945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=45373
--tserver_master_addrs=127.22.121.62:39221,127.22.121.61:39207,127.22.121.60:39647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:39.006997 4043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:39.008324 4043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:39.020820 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:39.020787 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:39.020782 4056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:39.021481 4043 server_base.cc:1034] running on GCE node
I20260811 06:17:39.022019 4043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:39.022864 4043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:39.024065 4043 hybrid_clock.cc:648] HybridClock initialized: now 1786429059024016 us; error 37 us; skew 500 ppm
I20260811 06:17:39.024341 4043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:39 dist-test-slave-gjrz krb5kdc[1882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:39.027254 4043 init.cc:377] Logged in from keytab as oryx/127.22.121.3@KRBTEST.COM (short username oryx)
I20260811 06:17:39.028517 4043 webserver.cc:460] Webserver started at http://127.22.121.3:45373/ using document root <none> and password file <none>
I20260811 06:17:39.029099 4043 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:39.029206 4043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:39.034042 4043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:17:39.036486 4071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:39.037665 4043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:39.037789 4043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "4622d1517748477bab51f2fa70971312"
format_stamp: "Formatted at 2026-08-11 06:17:35 on dist-test-slave-gjrz"
server_key: "e11c8c18128e53cd5c08e9f4c55b3e77"
server_key_iv: "2b5931cccc671701a36fa5a3c7c099c1"
server_key_version: "encryptionkey@0"
I20260811 06:17:39.038666 4043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:39.060505 4043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:39.064350 4043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:39.064606 4043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:39.065457 4043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:39.066704 4043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:39.066800 4043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:39.066905 4043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:39.066977 4043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:39.096573 4043 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42945
I20260811 06:17:39.096640 4276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42945 every 8 connection(s)
I20260811 06:17:39.097904 4043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:39.099176 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 4043
Aug 11 06:17:39 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.60@KRBTEST.COM
Aug 11 06:17:39 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.61@KRBTEST.COM
Aug 11 06:17:39 dist-test-slave-gjrz krb5kdc[1882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:39.165092 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.115728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37365 (local address 127.22.121.61:39207)
0811 06:17:39.140078 (+ 24350us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.140084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.141607 (+ 1523us) server_negotiation.cc:407] Connection header received
0811 06:17:39.141679 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.141684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.141765 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.141861 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.143247 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.144443 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.146639 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.147408 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.147509 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.154683 (+ 7174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.154691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.154696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.154788 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.159448 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.160656 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.160663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.160667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.160902 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.162621 (+ 1719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.162627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.162629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.162967 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.163095 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.164453 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.164697 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24228,"thread_start_us":24352,"threads_started":1}
I20260811 06:17:39.168151 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.115280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.22.121.3:37365)
0811 06:17:39.141204 (+ 25924us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.141220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.141322 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.142044 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.142049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.142755 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:39.143117 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.143130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.144792 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.144799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.146457 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.146473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.147795 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.147802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.148004 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.150178 (+ 2174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:39.150201 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:39.154420 (+ 4219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:39.159675 (+ 5255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.159685 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.159703 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.160467 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.162234 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.162239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.162243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.162500 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.163179 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:39.163188 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:39.163639 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.163957 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.166490 (+ 2533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25812,"thread_start_us":26093,"threads_started":1}
I20260811 06:17:39.169876 4282 heartbeater.cc:347] Connected to a master server at 127.22.121.61:39207
I20260811 06:17:39.170202 4282 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:39.171171 4282 heartbeater.cc:511] Master 127.22.121.61:39207 requested a full tablet report, sending...
I20260811 06:17:39.173039 3076 ts_manager.cc:194] Registered new tserver with Master: 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945)
I20260811 06:17:39.173877 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.100985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:36551 (local address 127.22.121.60:39647)
0811 06:17:39.130119 (+ 29134us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.130125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.134512 (+ 4387us) server_negotiation.cc:407] Connection header received
0811 06:17:39.134585 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.134589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.134683 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.134777 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.137442 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.138521 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.141093 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.141676 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.141765 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.162701 (+ 20936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.162707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.162711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.162747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.166088 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.167013 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.167018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.167021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.167170 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.171826 (+ 4656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.171830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.171834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.172219 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.172334 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.172841 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.173508 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":29015,"thread_start_us":33878,"threads_started":1}
I20260811 06:17:39.175866 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.101220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.22.121.3:36551)
0811 06:17:39.130725 (+ 29505us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.130784 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.133400 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.135180 (+ 1780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.135191 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.136104 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:39.137242 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.137267 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.138726 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.138731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.140884 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.140903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.142427 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.142435 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.142747 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.148683 (+ 5936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:39.148707 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:39.161209 (+ 12502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:39.166216 (+ 5007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.166219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.166222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.166865 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.167337 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.167341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.167344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.167510 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.172415 (+ 4905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:39.172420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:39.172565 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.174081 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.175160 (+ 1079us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29172,"mutex_wait_us":118,"spinlock_wait_cycles":13184,"thread_start_us":28947,"threads_started":1}
I20260811 06:17:39.176724 4279 heartbeater.cc:347] Connected to a master server at 127.22.121.60:39647
I20260811 06:17:39.176939 4279 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:39.177598 4279 heartbeater.cc:511] Master 127.22.121.60:39647 requested a full tablet report, sending...
I20260811 06:17:39.178169 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.100687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:48591 (local address 127.22.121.62:39221)
0811 06:17:39.129783 (+ 29096us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.129790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.134290 (+ 4500us) server_negotiation.cc:407] Connection header received
0811 06:17:39.134440 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.134447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.134539 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.134668 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.137487 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.138718 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.141087 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.141790 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.141899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.167706 (+ 25807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.167712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.167716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.167776 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.171038 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.174477 (+ 3439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.174482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.174486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.174664 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.176146 (+ 1482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.176152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.176156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.176580 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.176711 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.177405 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.177643 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28960,"thread_start_us":33690,"threads_started":1}
I20260811 06:17:39.179373 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.101775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:48591)
0811 06:17:39.134160 (+ 32385us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.134178 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.134301 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.135191 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.135196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.136132 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:39.137241 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.137267 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.138903 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.138908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.140880 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.140900 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.142428 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.142437 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.142753 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.143867 (+ 1114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:39.143899 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:39.166417 (+ 22518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:39.171174 (+ 4757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.171177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.171179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.171694 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.174815 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:39.174819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:39.174822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:39.174978 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:39.176797 (+ 1819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:39.176802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:39.177020 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.178416 (+ 1396us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.178696 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32259,"mutex_wait_us":206,"spinlock_wait_cycles":20864,"thread_start_us":32084,"threads_started":1}
I20260811 06:17:39.179400 3397 ts_manager.cc:194] Registered new tserver with Master: 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945)
I20260811 06:17:39.180238 4281 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39221
I20260811 06:17:39.180400 4281 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:39.181031 4281 heartbeater.cc:511] Master 127.22.121.62:39221 requested a full tablet report, sending...
I20260811 06:17:39.182266 1967 ts_manager.cc:194] Re-registered known tserver with Master: 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945)
I20260811 06:17:39.183490 1967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:48591
I20260811 06:17:39.187433 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.146164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.22.121.60:39647)
0811 06:17:39.158491 (+ 12327us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.158497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.161038 (+ 2541us) server_negotiation.cc:407] Connection header received
0811 06:17:39.161107 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.161112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.161185 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.161299 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.164875 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.166550 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.171512 (+ 4962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.172128 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.172249 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.177902 (+ 5653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.177912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.177917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.177953 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.183444 (+ 5491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.184392 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.184397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.184401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.184575 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.185040 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.185045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.185048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.185375 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.185623 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.186795 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.187000 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12215,"thread_start_us":12139,"threads_started":1}
I20260811 06:17:39.191725 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.158785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.22.121.61:39207)
0811 06:17:39.160668 (+ 1883us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.160674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.172099 (+ 11425us) server_negotiation.cc:407] Connection header received
0811 06:17:39.174464 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.174470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.174541 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.174621 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.177754 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.178987 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.181071 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.181737 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.181832 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.185361 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.185367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.185372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.185407 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.188700 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.189685 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.189689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.189691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.189834 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.190265 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.190269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.190270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.190539 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.190765 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.191178 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.191383 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1707,"thread_start_us":1525,"threads_started":1}
I20260811 06:17:39.195048 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.146161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.22.121.62:39221)
0811 06:17:39.172227 (+ 26066us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.172232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.172250 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:39.172298 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.172302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.172370 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.173401 (+ 1031us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.181707 (+ 8306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.183250 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.185136 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.185717 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.185827 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.189049 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.189056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.189060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.189101 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.192504 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.193230 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.193234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.193236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.193349 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.193727 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.193731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.193733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.193973 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.194185 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.194525 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.194729 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22732,"thread_start_us":23092,"threads_started":1}
I20260811 06:17:39.198558 1967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:39.206575 1967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:39.245983 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.226331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37120 (local address 127.22.121.3:42945)
0811 06:17:39.227121 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.227130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.236435 (+ 9305us) server_negotiation.cc:407] Connection header received
0811 06:17:39.236611 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.236621 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.236895 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.237126 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:39.238224 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.239744 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.243424 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.244860 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.244995 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.245189 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.245506 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.245663 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":390,"threads_started":1}
I20260811 06:17:39.246119 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.225078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33489 (local address 127.0.0.1:57714)
0811 06:17:39.225774 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.225792 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.225907 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.237583 (+ 11676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.237589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.237647 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:39.238268 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.238281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.240595 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.240602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.242787 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.242802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.245146 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.245153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.245372 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.245395 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.245496 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.245626 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":55,"spinlock_wait_cycles":18944,"thread_start_us":442,"threads_started":1}
I20260811 06:17:39.246435 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.225838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42945 (local address 127.0.0.1:37120)
0811 06:17:39.236323 (+ 10485us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.236340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.236466 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.237563 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.237568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.237593 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:39.238012 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.238022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.239908 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.239913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.243255 (+ 3342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.243295 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.245100 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.245106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.245309 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.245345 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.245452 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.245580 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10362,"mutex_wait_us":21,"spinlock_wait_cycles":5120,"thread_start_us":11546,"threads_started":1}
I20260811 06:17:39.247668 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.225471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.22.121.1:33489)
0811 06:17:39.236952 (+ 11481us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.236961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.236991 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:17:39.237091 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.237099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.237338 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.238474 (+ 1136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:39.238729 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.240411 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.242958 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.244643 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.244779 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.244987 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.246080 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.247259 (+ 1179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11291,"thread_start_us":11131,"threads_started":1}
I20260811 06:17:39.247936 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.227362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:42221 (local address 127.0.0.1:41060)
0811 06:17:39.237514 (+ 10152us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.237531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.237635 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.241540 (+ 3905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.241544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.241571 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:39.242209 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.242224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.243970 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.243976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.245430 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.245443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.247068 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.247085 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.247286 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.247320 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.247410 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.247526 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9996,"mutex_wait_us":23,"spinlock_wait_cycles":3328,"thread_start_us":8942,"threads_started":1}
I20260811 06:17:39.248163 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.226276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.22.121.2:42221)
0811 06:17:39.241088 (+ 14812us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.241096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.241121 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:17:39.241198 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.241203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.241357 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.241519 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:39.242426 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.243807 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.245594 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.246882 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.246995 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.247223 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.247452 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.247714 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14661,"thread_start_us":14548,"threads_started":1}
I20260811 06:17:39.250752 3621 tablet_service.cc:1467] Processing CreateTablet for tablet d75a0e08c8904c11aeae7866c42e09af (DEFAULT_TABLE table=test-table [id=1348d19ad76540ad9671d20e86d98231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:39.250805 4155 tablet_service.cc:1467] Processing CreateTablet for tablet d75a0e08c8904c11aeae7866c42e09af (DEFAULT_TABLE table=test-table [id=1348d19ad76540ad9671d20e86d98231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:39.252054 3621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75a0e08c8904c11aeae7866c42e09af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:39.251538 3907 tablet_service.cc:1467] Processing CreateTablet for tablet d75a0e08c8904c11aeae7866c42e09af (DEFAULT_TABLE table=test-table [id=1348d19ad76540ad9671d20e86d98231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:39.252300 4155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75a0e08c8904c11aeae7866c42e09af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:39.252768 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75a0e08c8904c11aeae7866c42e09af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:39.264144 4340 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Bootstrap starting.
I20260811 06:17:39.267189 4340 tablet_bootstrap.cc:654] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:39.268182 4341 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Bootstrap starting.
I20260811 06:17:39.268522 4340 log.cc:824] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:39.270437 4343 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Bootstrap starting.
I20260811 06:17:39.271924 4341 tablet_bootstrap.cc:654] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:39.273121 4341 log.cc:824] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:39.273701 4343 tablet_bootstrap.cc:654] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:39.274206 4340 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: No bootstrap required, opened a new log
I20260811 06:17:39.274487 4340 ts_tablet_manager.cc:1397] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 06:17:39.275059 4343 log.cc:824] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:39.276268 4341 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: No bootstrap required, opened a new log
I20260811 06:17:39.276523 4341 ts_tablet_manager.cc:1397] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260811 06:17:39.277751 4343 tablet_bootstrap.cc:492] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: No bootstrap required, opened a new log
I20260811 06:17:39.278090 4343 ts_tablet_manager.cc:1397] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 06:17:39.284739 4340 raft_consensus.cc:348] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.285180 4340 raft_consensus.cc:374] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:39.285286 4340 raft_consensus.cc:729] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc41c93cfcd4986a926a68e7410f94a, State: Initialized, Role: FOLLOWER
I20260811 06:17:39.285406 4343 raft_consensus.cc:348] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.285884 4343 raft_consensus.cc:374] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:39.285951 4340 consensus_queue.cc:260] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.286182 4343 raft_consensus.cc:729] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccaa050c07a48e49ebab022d0ee75d5, State: Initialized, Role: FOLLOWER
I20260811 06:17:39.286728 4341 raft_consensus.cc:348] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.286854 4343 consensus_queue.cc:260] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.287146 4341 raft_consensus.cc:374] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:39.287242 4341 raft_consensus.cc:729] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4622d1517748477bab51f2fa70971312, State: Initialized, Role: FOLLOWER
I20260811 06:17:39.287911 4341 consensus_queue.cc:260] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.289860 4341 ts_tablet_manager.cc:1428] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260811 06:17:39.290155 4281 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
I20260811 06:17:39.295352 4340 ts_tablet_manager.cc:1428] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Time spent starting tablet: real 0.021s user 0.016s sys 0.005s
I20260811 06:17:39.297654 4343 ts_tablet_manager.cc:1428] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
I20260811 06:17:39.298009 4020 heartbeater.cc:503] Master 127.22.121.62:39221 was elected leader, sending a full tablet report...
I20260811 06:17:39.362969 4352 raft_consensus.cc:482] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:39.363339 4352 raft_consensus.cc:504] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.365242 4352 leader_election.cc:288] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489), 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:39.371126 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.365560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:55419 (local address 127.22.121.1:33489)
0811 06:17:39.365758 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.365763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.365784 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:17:39.365964 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.365969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.366043 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.366125 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:39.366568 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.367371 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.369338 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.370152 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.370267 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.370357 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.370733 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.370861 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:17:39.371165 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.365435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33489 (local address 127.22.121.3:55419)
0811 06:17:39.365679 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.365698 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.365813 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.366180 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.366184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.366208 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:39.366430 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.366438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.367490 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.367495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.369151 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.369163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.370313 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.370319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.370538 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.370572 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.370681 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.370875 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":92}
I20260811 06:17:39.371214 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.365492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:42221 (local address 127.22.121.3:33549)
0811 06:17:39.365794 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:39.365812 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:39.365911 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:39.366200 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:39.366205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:39.366229 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:39.366427 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.366436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.367648 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.367651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.369271 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:39.369288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.370476 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.370480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:39.370629 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.370655 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:17:39.370746 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:17:39.370935 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":137}
I20260811 06:17:39.371147 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.365662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:33549 (local address 127.22.121.2:42221)
0811 06:17:39.365941 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.365947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.365964 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:17:39.366018 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.366021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.366075 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.366156 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:39.366577 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.367493 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.369418 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.370327 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.370462 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.370528 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.370797 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.370915 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260811 06:17:39.372215 3670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75a0e08c8904c11aeae7866c42e09af" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc41c93cfcd4986a926a68e7410f94a" is_pre_election: true
I20260811 06:17:39.372330 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75a0e08c8904c11aeae7866c42e09af" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" is_pre_election: true
I20260811 06:17:39.372699 3670 raft_consensus.cc:2454] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 0.
I20260811 06:17:39.372771 3941 raft_consensus.cc:2454] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 0.
I20260811 06:17:39.373322 4072 leader_election.cc:302] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc41c93cfcd4986a926a68e7410f94a, 4622d1517748477bab51f2fa70971312; no voters:
I20260811 06:17:39.373711 4352 raft_consensus.cc:2781] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:39.373864 4352 raft_consensus.cc:482] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:39.373963 4352 raft_consensus.cc:3037] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:39.376735 4352 raft_consensus.cc:504] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.377256 4352 leader_election.cc:288] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 election: Requested vote from peers 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489), 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:39.377663 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75a0e08c8904c11aeae7866c42e09af" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
I20260811 06:17:39.377674 3670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75a0e08c8904c11aeae7866c42e09af" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc41c93cfcd4986a926a68e7410f94a"
I20260811 06:17:39.377823 3941 raft_consensus.cc:3037] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:39.377821 3670 raft_consensus.cc:3037] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:39.379990 3670 raft_consensus.cc:2454] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 1.
I20260811 06:17:39.380213 3941 raft_consensus.cc:2454] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 1.
I20260811 06:17:39.380384 4072 leader_election.cc:302] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc41c93cfcd4986a926a68e7410f94a, 4622d1517748477bab51f2fa70971312; no voters:
I20260811 06:17:39.380753 4352 raft_consensus.cc:2781] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:39.381206 4352 raft_consensus.cc:686] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 1 LEADER]: Becoming Leader. State: Replica: 4622d1517748477bab51f2fa70971312, State: Running, Role: LEADER
I20260811 06:17:39.381946 4352 consensus_queue.cc:237] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } }
I20260811 06:17:39.386317 1966 catalog_manager.cc:5495] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4622d1517748477bab51f2fa70971312 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "4622d1517748477bab51f2fa70971312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } health_report { overall_health: UNKNOWN } } }
W20260811 06:17:39.412653 4296 tablet.cc:2369] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:39.424875 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.419400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.22.121.3:42945)
0811 06:17:39.419579 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.419584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.419619 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:17:39.419798 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.419801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.419882 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.420037 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:39.420392 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.421257 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.422532 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.423197 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.423338 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.423798 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:39.424394 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:39.424499 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.424587 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.424685 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:17:39.425845 3755 tablet.cc:2369] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:39.432794 3941 raft_consensus.cc:1261] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Refusing update from remote peer 4622d1517748477bab51f2fa70971312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:39.432791 3670 raft_consensus.cc:1261] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Refusing update from remote peer 4622d1517748477bab51f2fa70971312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:39.433830 4353 consensus_queue.cc:1035] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:39.434233 4352 consensus_queue.cc:1035] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:39.449513 4356 mvcc.cc:205] Tried to move back new op lower bound from 7317213427426824192 to 7317213427232026624. Current Snapshot: MvccSnapshot[applied={T|T < 7317213427426824192}]
I20260811 06:17:39.450986 4360 mvcc.cc:205] Tried to move back new op lower bound from 7317213427426824192 to 7317213427232026624. Current Snapshot: MvccSnapshot[applied={T|T < 7317213427426824192}]
I20260811 06:17:39.468638 1967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:39.468926 1967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:39.476137 1967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 477b028607914c79a70027d08838bbc2: Sending DeleteTablet for 3 replicas of tablet d75a0e08c8904c11aeae7866c42e09af
I20260811 06:17:39.477468 3621 tablet_service.cc:1514] Processing DeleteTablet for tablet d75a0e08c8904c11aeae7866c42e09af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:39 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:57714
I20260811 06:17:39.477500 3907 tablet_service.cc:1514] Processing DeleteTablet for tablet d75a0e08c8904c11aeae7866c42e09af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:39 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:41060
I20260811 06:17:39.477635 4155 tablet_service.cc:1514] Processing DeleteTablet for tablet d75a0e08c8904c11aeae7866c42e09af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:39 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:37120
I20260811 06:17:39.498397 4370 tablet_replica.cc:330] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: stopping tablet replica
I20260811 06:17:39.498946 4370 raft_consensus.cc:2229] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:39.500598 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 1915
I20260811 06:17:39.502422 4370 raft_consensus.cc:2258] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:39.503831 2020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:39.504005 4374 tablet_replica.cc:330] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: stopping tablet replica
I20260811 06:17:39.504923 4374 raft_consensus.cc:2229] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:39.505654 4374 raft_consensus.cc:2258] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:39.510656 4374 ts_tablet_manager.cc:1905] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:39.514423 4372 tablet_replica.cc:330] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: stopping tablet replica
I20260811 06:17:39.516520 4372 raft_consensus.cc:2229] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:39.517390 4372 raft_consensus.cc:2258] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:39.517442 4370 ts_tablet_manager.cc:1905] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:39.519593 4372 ts_tablet_manager.cc:1905] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:39.522671 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.480366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.22.121.61:39207)
0811 06:17:39.480569 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.480575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.499139 (+ 18564us) server_negotiation.cc:407] Connection header received
0811 06:17:39.500427 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.500432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.500530 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.500618 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.505854 (+ 5236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.506908 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.510864 (+ 3956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.511561 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.511648 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.515359 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.515365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.515370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.515410 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.519111 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.520007 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.520013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.520017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.520172 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.520625 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.520631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.520635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.521077 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.521357 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.521980 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.522200 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:39.526377 4374 ts_tablet_manager.cc:1918] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:39.526553 4374 log.cc:1196] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/d75a0e08c8904c11aeae7866c42e09af
I20260811 06:17:39.527088 4374 ts_tablet_manager.cc:1939] T d75a0e08c8904c11aeae7866c42e09af P 4622d1517748477bab51f2fa70971312: Deleting consensus metadata
I20260811 06:17:39.528815 4370 ts_tablet_manager.cc:1918] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:39.528977 4370 log.cc:1196] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/d75a0e08c8904c11aeae7866c42e09af
I20260811 06:17:39.529624 4370 ts_tablet_manager.cc:1939] T d75a0e08c8904c11aeae7866c42e09af P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting consensus metadata
I20260811 06:17:39.529678 4372 ts_tablet_manager.cc:1918] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:39.529965 4372 log.cc:1196] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/d75a0e08c8904c11aeae7866c42e09af
I20260811 06:17:39.530692 4372 ts_tablet_manager.cc:1939] T d75a0e08c8904c11aeae7866c42e09af P 1dc41c93cfcd4986a926a68e7410f94a: Deleting consensus metadata
I20260811 06:17:39.543028 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.499761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.22.121.60:39647)
0811 06:17:39.499962 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.499969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.523605 (+ 23636us) server_negotiation.cc:407] Connection header received
0811 06:17:39.523844 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.523849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.523952 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.524021 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.526137 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.527367 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.529805 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.530601 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.530707 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.534996 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.535003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.535019 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.535062 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.539387 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.540201 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.540207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.540211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.540435 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.540865 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.540870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.540873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.541283 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.541568 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.542117 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.542383 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:17:39.651731 1949 catalog_manager.cc:4841] TS 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221): tablet d75a0e08c8904c11aeae7866c42e09af (table test-table [id=1348d19ad76540ad9671d20e86d98231]) successfully deleted
I20260811 06:17:39.651731 1946 catalog_manager.cc:4841] TS 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945): tablet d75a0e08c8904c11aeae7866c42e09af (table test-table [id=1348d19ad76540ad9671d20e86d98231]) successfully deleted
W20260811 06:17:39.652092 1949 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.22.121.2:42221, user_credentials={real_user=slave}} blocked reactor thread for 120560us
W20260811 06:17:39.652186 1946 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.22.121.3:42945, user_credentials={real_user=slave}} blocked reactor thread for 123462us
I20260811 06:17:39.654824 1946 catalog_manager.cc:4841] TS 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489): tablet d75a0e08c8904c11aeae7866c42e09af (table test-table [id=1348d19ad76540ad9671d20e86d98231]) successfully deleted
I20260811 06:17:39.655063 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 1915
I20260811 06:17:39.711400 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.689552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.22.121.61:39207)
0811 06:17:39.689767 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.689773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.696149 (+ 6376us) server_negotiation.cc:407] Connection header received
0811 06:17:39.696452 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.696458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.696542 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.696633 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.698163 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.699117 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.700963 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.701433 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.701545 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.704918 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.704924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.704928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.704968 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.708527 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.709375 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.709379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.709382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.709510 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.709955 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.709960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.709962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.710291 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.710527 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.710878 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.711061 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 06:17:39.712741 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:39.689990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.22.121.60:39647)
0811 06:17:39.690150 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:39.690156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:39.696081 (+ 5925us) server_negotiation.cc:407] Connection header received
0811 06:17:39.696431 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:39.696436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:39.696521 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:39.696627 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:39.698666 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.699596 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.701169 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:39.701845 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:39.701988 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:39.704930 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:39.704936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:39.704940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:39.704977 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:39.708424 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.709121 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.709127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.709131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.709259 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:39.709903 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:39.709909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:39.709912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:39.710298 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:39.710529 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:39.712155 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0811 06:17:39.712385 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 06:17:40.460187 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.459388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:37905)
0811 06:17:40.459838 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.459976 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":401,"threads_started":1}
I20260811 06:17:40.460256 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.459018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:53513)
0811 06:17:40.459962 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.460090 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":729,"threads_started":1}
W20260811 06:17:40.460676 3750 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:39221 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
W20260811 06:17:40.460816 4020 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:39221 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:40.462888 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.461964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:51413)
0811 06:17:40.462465 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.462645 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":278,"threads_started":1}
W20260811 06:17:40.463227 4281 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:39221 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:40.975773 4394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 477b028607914c79a70027d08838bbc2)
I20260811 06:17:40.976040 4395 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 477b028607914c79a70027d08838bbc2)
I20260811 06:17:40.976037 4394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } }
I20260811 06:17:40.976274 4395 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } }
I20260811 06:17:40.977352 4394 leader_election.cc:288] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221), c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61:39207)
I20260811 06:17:40.977438 4395 leader_election.cc:288] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221), 08057ab7db9d4b34a88331bb3daddb30 (127.22.121.60:39647)
I20260811 06:17:40.990332 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.977516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37182)
0811 06:17:40.989857 (+ 12341us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.990053 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12237,"threads_started":1}
I20260811 06:17:40.991153 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.977509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37166)
0811 06:17:40.990660 (+ 13151us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.990886 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12650,"threads_started":1}
W20260811 06:17:40.992390 3375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
W20260811 06:17:40.992789 3056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:40.998970 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.998519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37190)
0811 06:17:40.998672 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.998823 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 06:17:40.999181 3056 leader_election.cc:334] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221): Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:41.000188 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.999580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37202)
0811 06:17:40.999851 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.000005 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 06:17:41.000383 3375 leader_election.cc:334] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221): Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:41.001642 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.977959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39207 (local address 127.0.0.1:60194)
0811 06:17:40.991351 (+ 13392us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.991369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:40.991504 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:40.995164 (+ 3660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:40.995170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:40.995201 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:40.995496 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:40.995507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:40.997246 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:40.997252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:40.999407 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:40.999421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.001001 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.001007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.001162 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.001189 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 06:17:41.001258 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 06:17:41.001415 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13286,"mutex_wait_us":49,"thread_start_us":12076,"threads_started":1}
I20260811 06:17:41.001776 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.989699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.22.121.61:39207)
0811 06:17:40.994638 (+ 4939us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:40.994644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:40.994665 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:17:40.994731 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:40.994736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:40.994808 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:40.994907 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:40.995761 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:40.997070 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:40.999565 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.000822 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.000921 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.000989 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.001332 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.001461 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":4820,"thread_start_us":5887,"threads_started":1}
I20260811 06:17:41.002317 3091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08057ab7db9d4b34a88331bb3daddb30" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" is_pre_election: true
I20260811 06:17:41.002683 3091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08057ab7db9d4b34a88331bb3daddb30 in term 1.
I20260811 06:17:41.003103 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.977906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:39647 (local address 127.0.0.1:55020)
0811 06:17:40.990517 (+ 12611us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:40.991442 (+ 925us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:40.991545 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:40.997311 (+ 5766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:40.997316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:40.997342 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:40.997602 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:40.997614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:40.999174 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:40.999179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.000940 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.000957 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.002404 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.002409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.002567 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.002602 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:17:41.002683 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:17:41.002772 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12520,"mutex_wait_us":19,"thread_start_us":11525,"threads_started":1}
I20260811 06:17:41.003166 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:40.978311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.22.121.60:39647)
0811 06:17:40.997021 (+ 18710us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:40.997027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:40.997047 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:17:40.997115 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:40.997121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:40.997190 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:40.997301 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:40.997814 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:40.999007 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.001097 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.002233 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.002366 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.002463 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.002732 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.002856 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18567,"thread_start_us":19253,"threads_started":1}
I20260811 06:17:41.003227 3376 leader_election.cc:302] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08057ab7db9d4b34a88331bb3daddb30, c3c75f4e3dbd477f8f713599130d3d8e; no voters: 477b028607914c79a70027d08838bbc2
I20260811 06:17:41.003661 4394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:17:41.003791 4394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 477b028607914c79a70027d08838bbc2)
I20260811 06:17:41.003715 3411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "08057ab7db9d4b34a88331bb3daddb30" is_pre_election: true
I20260811 06:17:41.003886 4394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:17:41.005932 4394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } }
I20260811 06:17:41.006453 3411 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3c75f4e3dbd477f8f713599130d3d8e in current term 2: Already voted for candidate 08057ab7db9d4b34a88331bb3daddb30 in this term.
I20260811 06:17:41.006623 4394 leader_election.cc:288] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 election: Requested vote from peers 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221), c3c75f4e3dbd477f8f713599130d3d8e (127.22.121.61:39207)
I20260811 06:17:41.007054 3091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08057ab7db9d4b34a88331bb3daddb30" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c3c75f4e3dbd477f8f713599130d3d8e"
I20260811 06:17:41.007135 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.006661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37216)
0811 06:17:41.006831 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.006938 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 06:17:41.007254 3091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:17:41.008168 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.007859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37224)
0811 06:17:41.008005 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.008077 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:17:41.008169 3053 leader_election.cc:302] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3c75f4e3dbd477f8f713599130d3d8e; no voters: 08057ab7db9d4b34a88331bb3daddb30, 477b028607914c79a70027d08838bbc2
W20260811 06:17:41.008343 3375 leader_election.cc:334] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221): Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
I20260811 06:17:41.009752 3091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08057ab7db9d4b34a88331bb3daddb30 in term 2.
I20260811 06:17:41.009918 4395 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:17:41.010157 3376 leader_election.cc:302] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08057ab7db9d4b34a88331bb3daddb30, c3c75f4e3dbd477f8f713599130d3d8e; no voters: 477b028607914c79a70027d08838bbc2
I20260811 06:17:41.010406 4394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:17:41.010876 4394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [term 2 LEADER]: Becoming Leader. State: Replica: 08057ab7db9d4b34a88331bb3daddb30, State: Running, Role: LEADER
I20260811 06:17:41.011248 4394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } }
I20260811 06:17:41.012902 4414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08057ab7db9d4b34a88331bb3daddb30. Latest consensus state: current_term: 2 leader_uuid: "08057ab7db9d4b34a88331bb3daddb30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:41.013232 4414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:41.013799 4419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:41.017910 4419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:41.018689 4419 catalog_manager.cc:1259] Loaded cluster ID: 64f9e51678b0499cab7d77115d58ec00
I20260811 06:17:41.018764 4419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:41.019881 4419 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:41.020635 4419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Loaded TSK: 0
I20260811 06:17:41.021442 4419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:17:41.077751 3395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:41.084612 3395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:41.088480 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.088098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37268)
0811 06:17:41.088254 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.088345 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 06:17:41.088632 3091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [term 2 FOLLOWER]: Refusing update from remote peer 08057ab7db9d4b34a88331bb3daddb30: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 06:17:41.089491 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.089120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37278)
0811 06:17:41.089278 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.089364 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 06:17:41.089413 4414 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 06:17:41.089700 3375 consensus_peers.cc:487] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 -> Peer 477b028607914c79a70027d08838bbc2 (127.22.121.62:39221): Couldn't send request to peer 477b028607914c79a70027d08838bbc2. Status: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:17:41.092017 4395 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08057ab7db9d4b34a88331bb3daddb30. Latest consensus state: current_term: 2 leader_uuid: "08057ab7db9d4b34a88331bb3daddb30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:41.092254 4395 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:17:41.093672 4414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "08057ab7db9d4b34a88331bb3daddb30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:41.093935 4414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30 [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:41.094587 4395 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "08057ab7db9d4b34a88331bb3daddb30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "477b028607914c79a70027d08838bbc2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39221 } } peers { permanent_uuid: "c3c75f4e3dbd477f8f713599130d3d8e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39207 } attrs { promote: false } } peers { permanent_uuid: "08057ab7db9d4b34a88331bb3daddb30" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 39647 } attrs { promote: false } } }
I20260811 06:17:41.094866 4395 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c75f4e3dbd477f8f713599130d3d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:17:41.109385 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.103062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.22.121.2:42221)
0811 06:17:41.103550 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.103556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.103573 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:17:41.103646 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.103651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.103729 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.104334 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:41.104504 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.105622 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.107720 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.108483 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.108600 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.108701 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.108984 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.109121 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":402,"threads_started":1}
I20260811 06:17:41.109392 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.102599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:42221 (local address 127.0.0.1:50636)
0811 06:17:41.102744 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.102760 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:41.102862 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:41.103886 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:41.103892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:41.103919 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:41.104197 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.104213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.105763 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.105769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.107592 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.107603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.108634 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.108639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.108829 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.108853 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:17:41.108935 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 06:17:41.109064 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":24}
I20260811 06:17:41.110013 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.102870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33489 (local address 127.0.0.1:42616)
0811 06:17:41.103022 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.103039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:41.103149 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:41.104099 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:41.104103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:41.104128 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:41.104452 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.104466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.105463 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.105468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.107505 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.107521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.109236 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.109244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.109437 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.109475 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:17:41.109547 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 06:17:41.109689 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260811 06:17:41.110311 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 7473249891074dccbb8d0f87ce0e809d (DEFAULT_TABLE table=test-table [id=3b2679febbef425f91bde40c1f0e9f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:41.110786 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.103338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.22.121.1:33489)
0811 06:17:41.103756 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.103761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.103781 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:17:41.103853 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.103857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.103931 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.104074 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:41.104610 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.105325 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.107667 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.108937 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.110222 (+ 1285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.110326 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.110378 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.110511 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":211,"threads_started":1}
I20260811 06:17:41.110782 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7473249891074dccbb8d0f87ce0e809d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:41.116014 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.103197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42945 (local address 127.0.0.1:59148)
0811 06:17:41.109757 (+ 6560us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.109773 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:41.109876 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:41.110271 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:41.110275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:41.110297 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:41.110456 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.110467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.111667 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.111672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.113489 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:41.113503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.115217 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.115224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:41.115495 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.115530 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:17:41.115618 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:17:41.115744 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6455,"threads_started":1}
I20260811 06:17:41.116043 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.103537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.22.121.3:42945)
0811 06:17:41.103975 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.103981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.109980 (+ 5999us) server_negotiation.cc:407] Connection header received
0811 06:17:41.110054 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.110059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.110131 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.110253 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:41.110613 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.111526 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.113660 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.115057 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.115189 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.115332 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.115655 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.115788 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":221,"threads_started":1}
I20260811 06:17:41.116721 4434 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Bootstrap starting.
I20260811 06:17:41.117044 4155 tablet_service.cc:1467] Processing CreateTablet for tablet 7473249891074dccbb8d0f87ce0e809d (DEFAULT_TABLE table=test-table [id=3b2679febbef425f91bde40c1f0e9f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:41.117319 3621 tablet_service.cc:1467] Processing CreateTablet for tablet 7473249891074dccbb8d0f87ce0e809d (DEFAULT_TABLE table=test-table [id=3b2679febbef425f91bde40c1f0e9f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:41.117444 4155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7473249891074dccbb8d0f87ce0e809d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:41.117671 3621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7473249891074dccbb8d0f87ce0e809d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:41.119181 4434 tablet_bootstrap.cc:654] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:41.121374 4434 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: No bootstrap required, opened a new log
I20260811 06:17:41.121526 4434 ts_tablet_manager.cc:1397] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:17:41.122285 4434 raft_consensus.cc:348] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.122558 4434 raft_consensus.cc:374] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:41.122646 4434 raft_consensus.cc:729] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccaa050c07a48e49ebab022d0ee75d5, State: Initialized, Role: FOLLOWER
I20260811 06:17:41.122890 4434 consensus_queue.cc:260] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.123673 4436 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Bootstrap starting.
I20260811 06:17:41.124995 4434 ts_tablet_manager.cc:1428] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 06:17:41.127650 4438 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Bootstrap starting.
I20260811 06:17:41.127832 4019 heartbeater.cc:503] Master 127.22.121.60:39647 was elected leader, sending a full tablet report...
I20260811 06:17:41.129331 4438 tablet_bootstrap.cc:654] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:41.130623 4436 tablet_bootstrap.cc:654] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:41.131491 4438 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: No bootstrap required, opened a new log
I20260811 06:17:41.131695 4438 ts_tablet_manager.cc:1397] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260811 06:17:41.132098 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.125464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:52171)
0811 06:17:41.131695 (+ 6231us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.131838 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6114,"thread_start_us":5984,"threads_started":1}
I20260811 06:17:41.132511 4438 raft_consensus.cc:348] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.132786 4438 raft_consensus.cc:374] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:41.132938 4438 raft_consensus.cc:729] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4622d1517748477bab51f2fa70971312, State: Initialized, Role: FOLLOWER
I20260811 06:17:41.133047 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.132643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:56921)
0811 06:17:41.132859 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.132937 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 06:17:41.133312 4436 tablet_bootstrap.cc:492] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: No bootstrap required, opened a new log
I20260811 06:17:41.133203 4438 consensus_queue.cc:260] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.133468 4436 ts_tablet_manager.cc:1397] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 06:17:41.133981 4438 ts_tablet_manager.cc:1428] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:41.135006 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.134159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:41229)
0811 06:17:41.134663 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.134777 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":248,"threads_started":1}
I20260811 06:17:41.135390 4436 raft_consensus.cc:348] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.135636 4436 raft_consensus.cc:374] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:41.135730 4436 raft_consensus.cc:729] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc41c93cfcd4986a926a68e7410f94a, State: Initialized, Role: FOLLOWER
I20260811 06:17:41.135752 4279 heartbeater.cc:503] Master 127.22.121.60:39647 was elected leader, sending a full tablet report...
I20260811 06:17:41.135962 4436 consensus_queue.cc:260] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.136305 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.135881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:56233)
0811 06:17:41.136082 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.136161 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 06:17:41.137130 4436 ts_tablet_manager.cc:1428] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:17:41.138332 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.137365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:39037)
0811 06:17:41.137929 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.138081 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":269,"threads_started":1}
I20260811 06:17:41.139300 3748 heartbeater.cc:503] Master 127.22.121.60:39647 was elected leader, sending a full tablet report...
I20260811 06:17:41.139532 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.139054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:49191)
0811 06:17:41.139326 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.139402 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260811 06:17:41.283643 4025 tablet.cc:2369] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:41.285588 4444 raft_consensus.cc:482] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:41.285806 4444 raft_consensus.cc:504] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.286579 4444 leader_election.cc:288] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489), 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:41.287168 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7473249891074dccbb8d0f87ce0e809d" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" is_pre_election: true
I20260811 06:17:41.287174 3670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7473249891074dccbb8d0f87ce0e809d" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc41c93cfcd4986a926a68e7410f94a" is_pre_election: true
I20260811 06:17:41.287422 3941 raft_consensus.cc:2454] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 0.
I20260811 06:17:41.287496 3670 raft_consensus.cc:2454] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 0.
I20260811 06:17:41.287883 4078 leader_election.cc:302] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4622d1517748477bab51f2fa70971312, 5ccaa050c07a48e49ebab022d0ee75d5; no voters:
I20260811 06:17:41.288195 4444 raft_consensus.cc:2781] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:41.288313 4444 raft_consensus.cc:482] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:41.288400 4444 raft_consensus.cc:3037] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:41.290441 4444 raft_consensus.cc:504] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.290894 4444 leader_election.cc:288] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 election: Requested vote from peers 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489), 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221)
I20260811 06:17:41.291404 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7473249891074dccbb8d0f87ce0e809d" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccaa050c07a48e49ebab022d0ee75d5"
I20260811 06:17:41.291445 3670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7473249891074dccbb8d0f87ce0e809d" candidate_uuid: "4622d1517748477bab51f2fa70971312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc41c93cfcd4986a926a68e7410f94a"
I20260811 06:17:41.291555 3941 raft_consensus.cc:3037] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:41.291610 3670 raft_consensus.cc:3037] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:41.293481 3941 raft_consensus.cc:2454] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 1.
I20260811 06:17:41.293617 3670 raft_consensus.cc:2454] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4622d1517748477bab51f2fa70971312 in term 1.
I20260811 06:17:41.293857 4078 leader_election.cc:302] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4622d1517748477bab51f2fa70971312, 5ccaa050c07a48e49ebab022d0ee75d5; no voters:
I20260811 06:17:41.294117 4444 raft_consensus.cc:2781] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:41.294512 4444 raft_consensus.cc:686] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 1 LEADER]: Becoming Leader. State: Replica: 4622d1517748477bab51f2fa70971312, State: Running, Role: LEADER
I20260811 06:17:41.294981 4444 consensus_queue.cc:237] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } }
I20260811 06:17:41.295629 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.295083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:34419)
0811 06:17:41.295299 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.295433 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:17:41.297603 3395 catalog_manager.cc:5495] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4622d1517748477bab51f2fa70971312 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "4622d1517748477bab51f2fa70971312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4622d1517748477bab51f2fa70971312" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42945 } health_report { overall_health: HEALTHY } } }
I20260811 06:17:41.357957 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.353254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.22.121.3:42945)
0811 06:17:41.353444 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.353450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.353469 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:17:41.353661 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.353664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.353730 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.353809 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:41.354220 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.354993 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.356364 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.356774 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.356888 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.357300 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:41.357402 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:41.357479 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.357644 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.357726 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:17:41.360816 3941 raft_consensus.cc:1261] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Refusing update from remote peer 4622d1517748477bab51f2fa70971312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:41.360872 3670 raft_consensus.cc:1261] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Refusing update from remote peer 4622d1517748477bab51f2fa70971312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:41.361490 4453 consensus_queue.cc:1035] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ccaa050c07a48e49ebab022d0ee75d5" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 42221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:41.361817 4444 consensus_queue.cc:1035] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc41c93cfcd4986a926a68e7410f94a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:41.364120 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.363664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:60523)
0811 06:17:41.363816 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.363931 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:17:41.364496 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.364027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:56621)
0811 06:17:41.364254 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.364360 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 06:17:41.373447 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.372784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:58629)
0811 06:17:41.373192 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.373287 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
I20260811 06:17:41.374404 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.374016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.3:34295)
0811 06:17:41.374178 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.374259 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 06:17:41.375126 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.374693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.2:52991)
0811 06:17:41.374894 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.374979 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 06:17:41.376241 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.375842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.22.121.1:52517)
0811 06:17:41.375996 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.376090 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:17:41.378656 3395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:41.378890 3395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:41.385537 3395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08057ab7db9d4b34a88331bb3daddb30: Sending DeleteTablet for 3 replicas of tablet 7473249891074dccbb8d0f87ce0e809d
I20260811 06:17:41.386595 3907 tablet_service.cc:1514] Processing DeleteTablet for tablet 7473249891074dccbb8d0f87ce0e809d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:41 UTC) from {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:50636
I20260811 06:17:41.386668 3621 tablet_service.cc:1514] Processing DeleteTablet for tablet 7473249891074dccbb8d0f87ce0e809d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:41 UTC) from {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:42616
I20260811 06:17:41.386794 3904 tablet_service.cc:1514] Processing DeleteTablet for tablet 7473249891074dccbb8d0f87ce0e809d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:41 UTC) from {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:50636
I20260811 06:17:41.387760 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 3443
I20260811 06:17:41.388080 4476 tablet_replica.cc:330] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: stopping tablet replica
I20260811 06:17:41.388247 4476 raft_consensus.cc:2229] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:41.388248 4155 tablet_service.cc:1514] Processing DeleteTablet for tablet 7473249891074dccbb8d0f87ce0e809d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:41 UTC) from {username='oryx', principal='oryx/127.22.121.60@KRBTEST.COM'} at 127.0.0.1:59148
I20260811 06:17:41.388522 4476 raft_consensus.cc:2258] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:41.389716 4476 ts_tablet_manager.cc:1905] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:41.390565 4477 tablet_replica.cc:330] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: stopping tablet replica
I20260811 06:17:41.390753 4477 raft_consensus.cc:2229] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:41.391045 4477 raft_consensus.cc:2258] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:41.392168 4477 ts_tablet_manager.cc:1905] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:41.394337 4476 ts_tablet_manager.cc:1918] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:41.394449 4476 log.cc:1196] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/7473249891074dccbb8d0f87ce0e809d
I20260811 06:17:41.394846 4476 ts_tablet_manager.cc:1939] T 7473249891074dccbb8d0f87ce0e809d P 5ccaa050c07a48e49ebab022d0ee75d5: Deleting consensus metadata
I20260811 06:17:41.395816 3375 catalog_manager.cc:4841] TS 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221): tablet 7473249891074dccbb8d0f87ce0e809d (table test-table [id=3b2679febbef425f91bde40c1f0e9f94]) successfully deleted
I20260811 06:17:41.396059 4478 tablet_replica.cc:330] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: stopping tablet replica
I20260811 06:17:41.396245 4478 raft_consensus.cc:2229] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 1 LEADER]: Raft consensus shutting down.
W20260811 06:17:41.396354 3375 catalog_manager.cc:4808] TS 5ccaa050c07a48e49ebab022d0ee75d5 (127.22.121.2:42221): delete failed for tablet 7473249891074dccbb8d0f87ce0e809d because the tablet was not found. No further retry: Not found: Tablet not found: 7473249891074dccbb8d0f87ce0e809d
I20260811 06:17:41.396608 4478 raft_consensus.cc:2258] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:41.397598 4478 ts_tablet_manager.cc:1905] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:41.398577 4477 ts_tablet_manager.cc:1918] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:41.398687 4477 log.cc:1196] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/7473249891074dccbb8d0f87ce0e809d
I20260811 06:17:41.399039 4477 ts_tablet_manager.cc:1939] T 7473249891074dccbb8d0f87ce0e809d P 1dc41c93cfcd4986a926a68e7410f94a: Deleting consensus metadata
I20260811 06:17:41.400055 3373 catalog_manager.cc:4841] TS 1dc41c93cfcd4986a926a68e7410f94a (127.22.121.1:33489): tablet 7473249891074dccbb8d0f87ce0e809d (table test-table [id=3b2679febbef425f91bde40c1f0e9f94]) successfully deleted
I20260811 06:17:41.403226 4478 ts_tablet_manager.cc:1918] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:41.403394 4478 log.cc:1196] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/7473249891074dccbb8d0f87ce0e809d
I20260811 06:17:41.403821 4478 ts_tablet_manager.cc:1939] T 7473249891074dccbb8d0f87ce0e809d P 4622d1517748477bab51f2fa70971312: Deleting consensus metadata
I20260811 06:17:41.404781 3373 catalog_manager.cc:4841] TS 4622d1517748477bab51f2fa70971312 (127.22.121.3:42945): tablet 7473249891074dccbb8d0f87ce0e809d (table test-table [id=3b2679febbef425f91bde40c1f0e9f94]) successfully deleted
I20260811 06:17:41.410313 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.388555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.22.121.1:33489)
0811 06:17:41.388715 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.388721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.397178 (+ 8457us) server_negotiation.cc:407] Connection header received
0811 06:17:41.397371 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.397377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.397471 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.397582 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:41.399317 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.400287 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.402003 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.402403 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.402489 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.405464 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:41.405475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:41.405490 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:41.405521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:41.408133 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.408713 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.408720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.408722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.408848 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.409126 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.409129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.409131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.409451 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:41.409663 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.409853 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.410012 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:17:41.410893 3735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:41.526854 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 3443
I20260811 06:17:41.548190 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 3773
I20260811 06:17:41.560642 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.548863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.22.121.2:42221)
0811 06:17:41.549032 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.549037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.549070 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:17:41.549246 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.549250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.549312 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.549392 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:41.550597 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.551311 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.552658 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.553073 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.553174 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.555709 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:41.555724 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:41.555742 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:41.555772 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:41.558455 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.559009 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.559016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.559019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.559149 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.559464 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.559467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.559470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.559810 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:41.560022 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.560234 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.560395 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:17:41.561290 4011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:41.642318 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.641566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37292)
0811 06:17:41.642012 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.642144 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":199,"threads_started":1}
I20260811 06:17:41.643603 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.643250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39221 (local address 127.0.0.1:37308)
0811 06:17:41.643408 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:41.643478 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:17:41.652415 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 3773
I20260811 06:17:41.675645 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 4043
I20260811 06:17:41.687675 4432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:41.676312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.22.121.3:42945)
0811 06:17:41.676491 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:41.676495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:41.676513 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:41.676649 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:41.676653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:41.676725 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:41.676805 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:41.678027 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.678717 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.679951 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:41.680342 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:41.680437 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:41.682719 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:41.682734 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:41.682751 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:41.682785 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:41.685393 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.685961 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.685967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.685970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.686123 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:41.686407 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:41.686410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:41.686412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:41.686737 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:41.686949 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:41.687152 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 06:17:41.687375 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:17:41.688331 4269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:41.803373 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 4043
I20260811 06:17:41.834681 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 2827
I20260811 06:17:41.835740 3128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:41.949393 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 2827
I20260811 06:17:41.968468 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 3002
I20260811 06:17:41.969517 3456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:42.091845 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 3002
2026-08-11T06:17:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9551 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:42 dist-test-slave-gjrz krb5kdc[4494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:42 dist-test-slave-gjrz krb5kdc[4494](info): set up 2 sockets
Aug 11 06:17:42 dist-test-slave-gjrz krb5kdc[4494](info): commencing operation
krb5kdc: starting...
W20260811 06:17:43.344399 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:43 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab.
Entry for principal oryx/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab.
I20260811 06:17:43.550158 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35353
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab
--principal=oryx/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:35353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:43.861765 4520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:43.862052 4520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:43.862185 4520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:43.869935 4520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:43.870054 4520 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:17:43.870121 4520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:43.870206 4520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:43.870275 4520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:43.870357 4520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:43.881542 4520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:35353
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.62.keytab
--principal=oryx/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:43.882659 4520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:43.883936 4520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:43.894559 4531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:43.894853 4532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:43.894896 4534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:43.896150 4520 server_base.cc:1034] running on GCE node
I20260811 06:17:43.896893 4520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:43.898094 4520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:43.899330 4520 hybrid_clock.cc:648] HybridClock initialized: now 1786429063899248 us; error 32 us; skew 500 ppm
I20260811 06:17:43.899712 4520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:43 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429063, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:43.902637 4520 init.cc:377] Logged in from keytab as oryx/127.22.121.62@KRBTEST.COM (short username oryx)
I20260811 06:17:43.903961 4520 webserver.cc:460] Webserver started at http://127.22.121.62:44597/ using document root <none> and password file <none>
I20260811 06:17:43.904608 4520 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:43.904706 4520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:43.905004 4520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:43.907851 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "ce66c752a039426eb2002dafc5f9a53d"
format_stamp: "Formatted at 2026-08-11 06:17:43 on dist-test-slave-gjrz"
server_key: "4bd9bf82d139ce8fe5d7aece78ddf7f2"
server_key_iv: "506e7ba162c6024f49cc1ecb7c3040b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:43.908588 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "ce66c752a039426eb2002dafc5f9a53d"
format_stamp: "Formatted at 2026-08-11 06:17:43 on dist-test-slave-gjrz"
server_key: "4bd9bf82d139ce8fe5d7aece78ddf7f2"
server_key_iv: "506e7ba162c6024f49cc1ecb7c3040b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:43.913570 4520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:17:43.916718 4547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:43.917881 4520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:43.918035 4520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "ce66c752a039426eb2002dafc5f9a53d"
format_stamp: "Formatted at 2026-08-11 06:17:43 on dist-test-slave-gjrz"
server_key: "4bd9bf82d139ce8fe5d7aece78ddf7f2"
server_key_iv: "506e7ba162c6024f49cc1ecb7c3040b2"
server_key_version: "encryptionkey@0"
I20260811 06:17:43.918216 4520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:43.931838 4520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:43.935176 4520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:43.935420 4520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:43.954901 4520 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:35353
I20260811 06:17:43.954922 4668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:35353 every 8 connection(s)
I20260811 06:17:43.956142 4520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:43.957861 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 4520
I20260811 06:17:43.958096 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:43.958915 23012 external_mini_cluster.cc:1428] Setting key 61f395a8fb13e4a5cffd84e452f7ddd8
I20260811 06:17:43.960527 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:17:43 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:43.973897 4669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Bootstrap starting.
I20260811 06:17:43.978036 4669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:43.979162 4669 log.cc:824] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:43.982266 4669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: No bootstrap required, opened a new log
I20260811 06:17:43.982372 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:43.960947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.22.121.62:35353)
0811 06:17:43.961671 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:43.961687 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:43.961736 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:17:43.963765 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:43.963793 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:43.964344 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:43.964859 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:43.966095 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:43.968117 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:43.969325 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:43.970296 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:43.970502 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:43.974228 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:43.974238 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:43.974254 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:43.974301 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:43.978158 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:43.978796 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:43.978805 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:43.978817 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:43.979022 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:43.979386 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:43.979391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:43.979394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:43.980019 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:43.980243 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:43.980742 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 06:17:43.981119 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":294,"threads_started":1}
I20260811 06:17:43.989506 4669 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } }
I20260811 06:17:43.989814 4669 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:43.989903 4669 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce66c752a039426eb2002dafc5f9a53d, State: Initialized, Role: FOLLOWER
I20260811 06:17:43.990360 4669 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } }
I20260811 06:17:43.990594 4669 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:43.990729 4669 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:43.990902 4669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:43.993044 4669 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } }
I20260811 06:17:43.993525 4669 leader_election.cc:302] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce66c752a039426eb2002dafc5f9a53d; no voters:
I20260811 06:17:43.994040 4669 leader_election.cc:288] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:43.994151 4681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:43.995172 4681 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [term 1 LEADER]: Becoming Leader. State: Replica: ce66c752a039426eb2002dafc5f9a53d, State: Running, Role: LEADER
I20260811 06:17:43.995751 4681 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } }
I20260811 06:17:43.996151 4669 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:43.998356 4684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce66c752a039426eb2002dafc5f9a53d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } } }
I20260811 06:17:43.998683 4684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:43.998530 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce66c752a039426eb2002dafc5f9a53d. Latest consensus state: current_term: 1 leader_uuid: "ce66c752a039426eb2002dafc5f9a53d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce66c752a039426eb2002dafc5f9a53d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35353 } } }
I20260811 06:17:43.998805 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:43.999583 4695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:44.005015 4695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:44.012969 4695 catalog_manager.cc:1347] Generated new cluster ID: ca159e1ae4024ceca5278fcb72b10499
I20260811 06:17:44.013079 4695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:44.023912 4695 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:44.025063 4695 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:44.034036 4695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Generated new TSK 0
I20260811 06:17:44.034795 4695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
I20260811 06:17:44.099545 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35353
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:44.412428 4718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:44.412680 4718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:44.412783 4718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:44.419556 4718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:44.419648 4718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:44.419731 4718 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:17:44.419807 4718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:44.428974 4718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:44.430088 4718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:44.431285 4718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:44.443423 4725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:44.443559 4723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:44.443524 4728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:44.444130 4718 server_base.cc:1034] running on GCE node
I20260811 06:17:44.444718 4718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:44.445565 4718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:44.446774 4718 hybrid_clock.cc:648] HybridClock initialized: now 1786429064446709 us; error 40 us; skew 500 ppm
I20260811 06:17:44.447041 4718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:44 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429064, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:44.450006 4718 init.cc:377] Logged in from keytab as oryx/127.22.121.1@KRBTEST.COM (short username oryx)
I20260811 06:17:44.451419 4718 webserver.cc:460] Webserver started at http://127.22.121.1:33681/ using document root <none> and password file <none>
I20260811 06:17:44.452054 4718 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:44.452152 4718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:44.452441 4718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:44.455129 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "9e8bf948919249aca93b16b7a4cdddea"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "620d5544dab69b9dbc437bd0d5d81499"
server_key_iv: "c045836af203d443b67d5c4e43979720"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.455932 4718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "9e8bf948919249aca93b16b7a4cdddea"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "620d5544dab69b9dbc437bd0d5d81499"
server_key_iv: "c045836af203d443b67d5c4e43979720"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.460642 4718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:17:44.463871 4743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:44.465019 4718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260811 06:17:44.465163 4718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "9e8bf948919249aca93b16b7a4cdddea"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "620d5544dab69b9dbc437bd0d5d81499"
server_key_iv: "c045836af203d443b67d5c4e43979720"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.465344 4718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:44.486958 4718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:44.490893 4718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:44.491128 4718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:44.493688 4718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:44.493822 4718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:44.493969 4718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:44.494055 4718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:17:44 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429064, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:44.515822 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.493983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:40465 (local address 127.22.121.62:35353)
0811 06:17:44.494452 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:44.494459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:44.495582 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 06:17:44.497756 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:44.497761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:44.497847 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:44.497946 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:44.499950 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.501088 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.503624 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.504025 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.504141 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.508822 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:44.508829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:44.508833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:44.508876 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:44.512297 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:44.513267 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:44.513272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:44.513276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:44.513433 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:44.513985 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:44.513991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:44.513995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:44.514387 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:44.514560 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:44.515171 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 06:17:44.515435 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":241,"threads_started":1}
I20260811 06:17:44.516656 4757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.494640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.1:40465)
0811 06:17:44.495430 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:44.495468 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:44.497330 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:44.498205 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:44.498213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:44.498905 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:44.499746 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.499782 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.501298 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.501304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.503429 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.503445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.504293 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.504302 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.504597 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.505355 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:44.505376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:44.508522 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:44.512462 (+ 3940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.512471 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.512485 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:44.513082 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:44.513532 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.513537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.513541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:44.513826 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:44.514517 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:44.514523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:44.514823 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 06:17:44.515040 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 06:17:44.515470 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":307,"threads_started":1}
I20260811 06:17:44.534614 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.521954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:45803 (local address 127.22.121.62:35353)
0811 06:17:44.522148 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:44.522154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:44.522775 (+ 621us) server_negotiation.cc:407] Connection header received
0811 06:17:44.522920 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:44.522924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:44.522981 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:44.523100 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:44.524316 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.525166 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.527056 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.527636 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.527747 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.529326 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:44.529330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:44.529333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:44.529362 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:44.531753 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:44.532600 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:44.532605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:44.532609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:44.532740 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:44.533087 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:44.533093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:44.533096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:44.533452 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:44.533608 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:44.534045 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 06:17:44.534271 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:44.534796 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.521966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.1:45803)
0811 06:17:44.522671 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:44.522691 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:44.522826 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:44.523129 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:44.523134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:44.523751 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:44.524170 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.524181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.525340 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.525345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.526856 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.526875 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.527822 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.527828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.527960 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.528464 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:44.528481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:44.529156 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:44.531922 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.531927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.531931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:44.532479 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:44.532855 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.532858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.532861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:44.532982 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:44.533554 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:44.533559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:44.533780 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 06:17:44.533930 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:17:44.534275 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":356,"threads_started":1}
I20260811 06:17:44.540699 4718 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:36481
I20260811 06:17:44.540731 4964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:36481 every 8 connection(s)
I20260811 06:17:44.541826 4718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:44.543658 4965 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35353
I20260811 06:17:44.543862 4965 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:44.544435 4965 heartbeater.cc:511] Master 127.22.121.62:35353 requested a full tablet report, sending...
I20260811 06:17:44.546231 4580 ts_manager.cc:194] Registered new tserver with Master: 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481)
I20260811 06:17:44.547631 4580 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:40465
I20260811 06:17:44.548740 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 4718
I20260811 06:17:44.548923 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:44.549511 23012 external_mini_cluster.cc:1428] Setting key 48277f6ef09cb1b7966951fafff23eb3
WARNING: no policy specified for oryx/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
I20260811 06:17:44.594151 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35353
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:44.899420 4970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:44.899672 4970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:44.899776 4970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:44.906670 4970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:44.906761 4970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:44.906847 4970 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:17:44.906921 4970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:44.916183 4970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:44.917224 4970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:44.918412 4970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:44.930902 4981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:44.930855 4985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:44.930822 4982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:44.931568 4970 server_base.cc:1034] running on GCE node
I20260811 06:17:44.932118 4970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:44.932938 4970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:44.934159 4970 hybrid_clock.cc:648] HybridClock initialized: now 1786429064934093 us; error 52 us; skew 500 ppm
I20260811 06:17:44.934435 4970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:44 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429064, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:44.937358 4970 init.cc:377] Logged in from keytab as oryx/127.22.121.2@KRBTEST.COM (short username oryx)
I20260811 06:17:44.938699 4970 webserver.cc:460] Webserver started at http://127.22.121.2:38791/ using document root <none> and password file <none>
I20260811 06:17:44.939381 4970 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:44.939563 4970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:44.939886 4970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:44.942679 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "4125d19587154305bd262883bf3aab48"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "7755df5b667abbc70eb2d4a97c97f61d"
server_key_iv: "86ed9291ded381e3d53a38777d92f665"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.943558 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "4125d19587154305bd262883bf3aab48"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "7755df5b667abbc70eb2d4a97c97f61d"
server_key_iv: "86ed9291ded381e3d53a38777d92f665"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.948494 4970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 06:17:44.951879 4997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:44.953109 4970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:17:44.953260 4970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "4125d19587154305bd262883bf3aab48"
format_stamp: "Formatted at 2026-08-11 06:17:44 on dist-test-slave-gjrz"
server_key: "7755df5b667abbc70eb2d4a97c97f61d"
server_key_iv: "86ed9291ded381e3d53a38777d92f665"
server_key_version: "encryptionkey@0"
I20260811 06:17:44.953464 4970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:44.973264 4970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:44.976964 4970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:44.977226 4970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:44.979805 4970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:44.979911 4970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:44.980078 4970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:44.980139 4970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:17:44 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429064, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:45.002126 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.980199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:58513 (local address 127.22.121.62:35353)
0811 06:17:44.980386 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:44.980394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:44.981870 (+ 1476us) server_negotiation.cc:407] Connection header received
0811 06:17:44.984607 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:44.984613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:44.984698 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:44.984802 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:44.986814 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.987989 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.990302 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.990972 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.991121 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.995251 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:44.995258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:44.995283 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:44.995325 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:44.998764 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:44.999562 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:44.999567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:44.999571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:44.999759 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.000370 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.000375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.000378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.000692 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:45.000843 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:45.001518 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 06:17:45.001738 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:17:45.003012 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:44.980780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.2:58513)
0811 06:17:44.981673 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:44.981722 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:44.984234 (+ 2512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:44.985096 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:44.985108 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:44.985829 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:44.986591 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.986621 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.988182 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.988190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.990138 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:44.990151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:44.991244 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:44.991252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:44.991459 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:44.992115 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:44.992156 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:44.995010 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:44.998915 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.998921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.998932 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:44.999438 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:44.999916 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:44.999926 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:44.999931 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.000210 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.000837 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:45.000844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:45.001165 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 06:17:45.001389 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 06:17:45.001867 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":417,"threads_started":1}
I20260811 06:17:45.030102 4970 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:32843
I20260811 06:17:45.030180 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.009061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49835 (local address 127.22.121.62:35353)
0811 06:17:45.009192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:45.009197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:45.016126 (+ 6929us) server_negotiation.cc:407] Connection header received
0811 06:17:45.016292 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:45.016298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:45.016378 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:45.016492 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:45.017548 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.018668 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.020734 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.021410 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.021540 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.023597 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:45.023603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:45.023608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:45.023643 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:45.026981 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.027888 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.027894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.027897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.028061 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.028448 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.028453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.028456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.028913 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:45.029064 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:45.029551 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 06:17:45.029780 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 06:17:45.030149 5195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:32843 every 8 connection(s)
I20260811 06:17:45.030298 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.009374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.2:49835)
0811 06:17:45.016002 (+ 6628us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:45.016022 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:45.016167 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:45.016534 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:45.016539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:45.017137 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:45.017410 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.017419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.018815 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.018820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.020582 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.020598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.021611 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.021618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.021795 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.022459 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:45.022475 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:45.023390 (+ 915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:45.027138 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.027143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.027150 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.027747 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.028185 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.028194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.028197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.028327 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.029048 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:45.029057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:45.029249 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 06:17:45.029413 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 06:17:45.029775 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6496,"thread_start_us":6383,"threads_started":1}
I20260811 06:17:45.031401 4970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:45.033360 5198 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35353
I20260811 06:17:45.033736 5198 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:45.034431 5198 heartbeater.cc:511] Master 127.22.121.62:35353 requested a full tablet report, sending...
I20260811 06:17:45.035676 4580 ts_manager.cc:194] Registered new tserver with Master: 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843)
I20260811 06:17:45.036389 4580 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:58513
I20260811 06:17:45.040458 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 4970
I20260811 06:17:45.040619 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:45.040982 23012 external_mini_cluster.cc:1428] Setting key 5d7ff5714c5091ed2498fe8356bddc37
WARNING: no policy specified for oryx/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
I20260811 06:17:45.087069 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35353
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:45.397502 5205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:45.397754 5205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:45.397881 5205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:45.404814 5205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:45.404923 5205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:45.405033 5205 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:17:45.405121 5205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:45.414182 5205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.121.62:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:45.415307 5205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:45.416507 5205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:45.428957 5214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:45.429032 5222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:45.429006 5219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:45.429898 5205 server_base.cc:1034] running on GCE node
I20260811 06:17:45.430404 5205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:45.431166 5205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:45.432376 5205 hybrid_clock.cc:648] HybridClock initialized: now 1786429065432319 us; error 35 us; skew 500 ppm
I20260811 06:17:45.432649 5205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:45 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429065, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:45.435568 5205 init.cc:377] Logged in from keytab as oryx/127.22.121.3@KRBTEST.COM (short username oryx)
I20260811 06:17:45.436875 5205 webserver.cc:460] Webserver started at http://127.22.121.3:43179/ using document root <none> and password file <none>
I20260811 06:17:45.437489 5205 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:45.437578 5205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:45.437853 5205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:45.440802 5205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "2c019fd706b74eb083116074b119b8b9"
format_stamp: "Formatted at 2026-08-11 06:17:45 on dist-test-slave-gjrz"
server_key: "81b4212d13bbc6c187ae28a7ad22bb53"
server_key_iv: "953dd0096d2fc2a4c6e749740e06929f"
server_key_version: "encryptionkey@0"
I20260811 06:17:45.441511 5205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "2c019fd706b74eb083116074b119b8b9"
format_stamp: "Formatted at 2026-08-11 06:17:45 on dist-test-slave-gjrz"
server_key: "81b4212d13bbc6c187ae28a7ad22bb53"
server_key_iv: "953dd0096d2fc2a4c6e749740e06929f"
server_key_version: "encryptionkey@0"
I20260811 06:17:45.446216 5205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:17:45.449412 5233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:45.450570 5205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:45.450717 5205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "2c019fd706b74eb083116074b119b8b9"
format_stamp: "Formatted at 2026-08-11 06:17:45 on dist-test-slave-gjrz"
server_key: "81b4212d13bbc6c187ae28a7ad22bb53"
server_key_iv: "953dd0096d2fc2a4c6e749740e06929f"
server_key_version: "encryptionkey@0"
I20260811 06:17:45.450902 5205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:45.482517 5205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:45.486011 5205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:45.486249 5205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:45.488744 5205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:45.488844 5205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:45.489012 5205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:45.489135 5205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 06:17:45 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429065, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:45.510530 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.489155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:39071 (local address 127.22.121.62:35353)
0811 06:17:45.489320 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:45.489325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:45.490688 (+ 1363us) server_negotiation.cc:407] Connection header received
0811 06:17:45.493587 (+ 2899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:45.493618 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:45.493731 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:45.493846 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:45.496011 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.497116 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.499778 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.500387 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.500507 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.504855 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:45.504860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:45.504863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:45.504895 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:45.507453 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.508197 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.508202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.508206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.508379 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.508912 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.508915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.508917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.509182 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:45.509299 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:45.510127 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 06:17:45.510289 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:17:45.512226 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.489798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.3:39071)
0811 06:17:45.490540 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:45.490579 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:45.493251 (+ 2672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:45.494214 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:45.494226 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:45.494892 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:45.495777 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.495807 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.497321 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.497326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.499592 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.499611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.500721 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.500727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.500978 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.501620 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:45.501654 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:45.504649 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:45.507597 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.507609 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.507624 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.508066 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.508511 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.508516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.508524 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.508796 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.509287 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:45.509295 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:45.509719 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 06:17:45.510001 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:17:45.510505 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":337,"threads_started":1}
I20260811 06:17:45.532406 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.517969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.22.121.3:55545)
0811 06:17:45.518565 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:45.518583 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:45.518706 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:45.518994 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:45.518998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:45.519494 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:45.519930 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.519944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.521049 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.521054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.522914 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:45.522930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.523844 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.523852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:45.524141 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.524806 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:45.524832 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:45.526007 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:45.529782 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.529787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.529791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.530268 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.530719 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:45.530725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:45.530730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:45.530902 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:45.531570 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:45.531573 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:45.531713 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 06:17:45.531838 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 06:17:45.532072 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":303,"threads_started":1}
I20260811 06:17:45.532567 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:45.517917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:55545 (local address 127.22.121.62:35353)
0811 06:17:45.518099 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:45.518103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:45.518661 (+ 558us) server_negotiation.cc:407] Connection header received
0811 06:17:45.518812 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:45.518818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:45.518880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:45.518958 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:45.520078 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.520926 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.523065 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.523609 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.523735 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:45.526250 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:45.526256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:45.526261 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:45.526298 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:45.529649 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.530382 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.530388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.530393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.530570 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:45.531057 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:45.531062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:45.531066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:45.531460 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:45.531603 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:45.531982 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 06:17:45.532208 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:17:45.539927 5205 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:34641
I20260811 06:17:45.539968 5448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:34641 every 8 connection(s)
I20260811 06:17:45.544365 5205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:45.544742 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 5205
I20260811 06:17:45.544961 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:45.545463 23012 external_mini_cluster.cc:1428] Setting key ab9e0b073991ecebad84028d87089179
I20260811 06:17:45.546752 5450 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35353
I20260811 06:17:45.546969 5450 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:45.547609 5450 heartbeater.cc:511] Master 127.22.121.62:35353 requested a full tablet report, sending...
I20260811 06:17:45.548789 4574 ts_manager.cc:194] Registered new tserver with Master: 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:45.549572 4574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:39071
I20260811 06:17:45.550058 4965 heartbeater.cc:503] Master 127.22.121.62:35353 was elected leader, sending a full tablet report...
I20260811 06:17:45.560223 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:17:45 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429065, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:17:45 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:45 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:46.038513 5198 heartbeater.cc:503] Master 127.22.121.62:35353 was elected leader, sending a full tablet report...
I20260811 06:17:46.552135 5450 heartbeater.cc:503] Master 127.22.121.62:35353 was elected leader, sending a full tablet report...
W20260811 06:17:46.691413 5464 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.087s user 0.001s sys 0.014s
W20260811 06:17:46.692677 4754 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:45.599296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.22.121.62:35353)
0811 06:17:45.599499 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:45.599505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:45.599737 (+ 232us) server_negotiation.cc:407] Connection header received
0811 06:17:45.599893 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:45.599897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:45.599961 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:45.600031 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:45.601250 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.601993 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.603124 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:45.603571 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:45.603668 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.692340 (+1088672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36300: BlockingRecv error: recv got EOF from 127.0.0.1:36300 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:17:46 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429065, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:46.713941 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.696605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.22.121.62:35353)
0811 06:17:46.696768 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:46.696772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:46.697129 (+ 357us) server_negotiation.cc:407] Connection header received
0811 06:17:46.697306 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:46.697310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:46.697402 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:46.697493 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:46.698664 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.699438 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.700684 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.701149 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.701249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.709056 (+ 7807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:46.709061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:46.709064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:46.709108 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:46.711735 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:46.712361 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:46.712365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:46.712367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:46.712488 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:46.712837 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:46.712840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:46.712842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:46.713092 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:46.713321 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:46.713495 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 06:17:46.713714 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:17:46.718149 4574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:46.725132 4574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:46.758361 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.737187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.22.121.2:32843)
0811 06:17:46.748736 (+ 11549us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:46.748745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:46.748776 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:17:46.748882 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:46.748889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:46.749141 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:46.749331 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:46.750775 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.752430 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.755326 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.756627 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.756798 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.756999 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:46.757904 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 06:17:46.758066 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11384,"thread_start_us":11231,"threads_started":1}
I20260811 06:17:46.758397 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.736665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32843 (local address 127.0.0.1:52712)
0811 06:17:46.747953 (+ 11288us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:46.747996 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:46.748174 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:46.750000 (+ 1826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:46.750008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:46.750058 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:46.750545 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.750559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.752628 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.752635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.755141 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.755156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.756854 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.756860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.757057 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.757113 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 06:17:46.757304 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 06:17:46.757483 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10354,"thread_start_us":17367,"threads_started":1}
I20260811 06:17:46.762140 5087 tablet_service.cc:1467] Processing CreateTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 (DEFAULT_TABLE table=test-table [id=9d0d8b6bc4994acebac25d6392926b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:46.762892 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.736951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.0.0.1:33856)
0811 06:17:46.755217 (+ 18266us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:46.755239 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:46.755371 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:46.755928 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:46.755932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:46.755956 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:46.756300 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.756311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.758202 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.758207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.760384 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.760399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.762143 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.762149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.762303 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.762333 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:17:46.762430 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 06:17:46.762569 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18144,"thread_start_us":17901,"threads_started":1}
I20260811 06:17:46.763231 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.737242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.22.121.3:34641)
0811 06:17:46.749879 (+ 12637us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:46.749890 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:46.755354 (+ 5464us) server_negotiation.cc:407] Connection header received
0811 06:17:46.755512 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:46.755520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:46.755758 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:46.755934 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:46.756504 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.758028 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.760550 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.761934 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.762072 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.762271 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:46.762480 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 06:17:46.762651 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12485,"thread_start_us":12363,"threads_started":1}
I20260811 06:17:46.763314 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.737732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.22.121.1:36481)
0811 06:17:46.754566 (+ 16834us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:46.754575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:46.755716 (+ 1141us) server_negotiation.cc:407] Connection header received
0811 06:17:46.755860 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:46.755866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:46.756063 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:46.756263 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:46.756870 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.758310 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.760742 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.762049 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.762180 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.762378 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:46.762885 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 06:17:46.763058 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16681,"thread_start_us":16837,"threads_started":1}
I20260811 06:17:46.763756 5087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd9f752fd5454ac8befd66f9cb3d6727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:46.763899 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:46.737254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36481 (local address 127.0.0.1:60342)
0811 06:17:46.755592 (+ 18338us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:46.755613 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:46.755720 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:46.756265 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:46.756269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:46.756295 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:46.756659 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.756671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.758499 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.758519 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.760597 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:46.760608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:46.762515 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:46.762522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:46.762682 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:46.762715 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:17:46.763540 (+ 825us) client_negotiation.cc:240] Negotiation successful
0811 06:17:46.763634 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18228,"mutex_wait_us":3,"thread_start_us":10028,"threads_started":1}
I20260811 06:17:46.766638 5330 tablet_service.cc:1467] Processing CreateTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 (DEFAULT_TABLE table=test-table [id=9d0d8b6bc4994acebac25d6392926b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:46.767169 4838 tablet_service.cc:1467] Processing CreateTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 (DEFAULT_TABLE table=test-table [id=9d0d8b6bc4994acebac25d6392926b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:46.767767 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd9f752fd5454ac8befd66f9cb3d6727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:46.768532 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd9f752fd5454ac8befd66f9cb3d6727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:46.777985 5497 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Bootstrap starting.
I20260811 06:17:46.778810 5499 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Bootstrap starting.
I20260811 06:17:46.781026 5497 tablet_bootstrap.cc:654] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:46.782018 5497 log.cc:824] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:46.782037 5499 tablet_bootstrap.cc:654] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:46.783246 5499 log.cc:824] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:46.784806 5501 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Bootstrap starting.
I20260811 06:17:46.786170 5499 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: No bootstrap required, opened a new log
I20260811 06:17:46.786392 5499 ts_tablet_manager.cc:1397] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 06:17:46.787428 5501 tablet_bootstrap.cc:654] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:46.788327 5501 log.cc:824] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:46.789003 5497 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: No bootstrap required, opened a new log
I20260811 06:17:46.789258 5497 ts_tablet_manager.cc:1397] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 06:17:46.790511 5501 tablet_bootstrap.cc:492] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: No bootstrap required, opened a new log
I20260811 06:17:46.790692 5501 ts_tablet_manager.cc:1397] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 06:17:46.795984 5499 raft_consensus.cc:348] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.796459 5499 raft_consensus.cc:374] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:46.796558 5499 raft_consensus.cc:729] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:46.797148 5499 consensus_queue.cc:260] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.798344 5501 raft_consensus.cc:348] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.798641 5501 raft_consensus.cc:374] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:46.798754 5501 raft_consensus.cc:729] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Initialized, Role: FOLLOWER
I20260811 06:17:46.799081 5497 raft_consensus.cc:348] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.799364 5501 consensus_queue.cc:260] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.799563 5497 raft_consensus.cc:374] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:46.799733 5497 raft_consensus.cc:729] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125d19587154305bd262883bf3aab48, State: Initialized, Role: FOLLOWER
I20260811 06:17:46.800163 5499 ts_tablet_manager.cc:1428] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260811 06:17:46.800319 5497 consensus_queue.cc:260] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:46.802093 5497 ts_tablet_manager.cc:1428] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 06:17:46.803747 5501 ts_tablet_manager.cc:1428] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
W20260811 06:17:47.034760 5199 tablet.cc:2369] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:17:47.044777 4966 tablet.cc:2369] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:17:47.046763 5452 tablet.cc:2369] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:17:47.095098 5510 raft_consensus.cc:482] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:47.095424 5510 raft_consensus.cc:504] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:47.097223 5510 leader_election.cc:288] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641), 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481)
I20260811 06:17:47.103842 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.097768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:33859 (local address 127.22.121.3:34641)
0811 06:17:47.097980 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.097986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.098557 (+ 571us) server_negotiation.cc:407] Connection header received
0811 06:17:47.098735 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.098740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.098812 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.098889 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.099649 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.100390 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.102183 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.102935 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.103077 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.103151 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.103491 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.103616 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:17:47.103885 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.097652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49289 (local address 127.22.121.1:36481)
0811 06:17:47.097860 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.097868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.098533 (+ 665us) server_negotiation.cc:407] Connection header received
0811 06:17:47.098727 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.098731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.098803 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.098881 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.099568 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.100310 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.102198 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.102940 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.103090 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.103164 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.103524 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.103660 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:17:47.103999 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.097656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.22.121.2:33859)
0811 06:17:47.098454 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.098469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.098604 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.098913 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.098916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.099004 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.099473 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.099486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.100511 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.100517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.102035 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.102047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.103152 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.103158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.103334 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.103363 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.103440 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.103558 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":28,"spinlock_wait_cycles":7168,"thread_start_us":494,"threads_started":1}
I20260811 06:17:47.104022 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.097618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36481 (local address 127.22.121.2:49289)
0811 06:17:47.098419 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.098435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.098590 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.098911 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.098915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.098960 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.099362 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.099372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.100443 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.100448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.102040 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.102053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.103207 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.103214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.103381 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.103404 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.103483 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.103608 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":7,"spinlock_wait_cycles":18048,"thread_start_us":421,"threads_started":1}
I20260811 06:17:47.104931 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd9f752fd5454ac8befd66f9cb3d6727" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9" is_pre_election: true
I20260811 06:17:47.105479 5370 raft_consensus.cc:2454] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 0.
I20260811 06:17:47.105445 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd9f752fd5454ac8befd66f9cb3d6727" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea" is_pre_election: true
I20260811 06:17:47.105952 4889 raft_consensus.cc:2454] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 0.
I20260811 06:17:47.106108 5004 leader_election.cc:302] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 4125d19587154305bd262883bf3aab48; no voters:
I20260811 06:17:47.106467 5510 raft_consensus.cc:2781] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:47.106603 5510 raft_consensus.cc:482] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:47.106693 5510 raft_consensus.cc:3037] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.108834 5510 raft_consensus.cc:504] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:47.109375 5510 leader_election.cc:288] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 election: Requested vote from peers 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641), 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481)
I20260811 06:17:47.109753 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd9f752fd5454ac8befd66f9cb3d6727" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9"
I20260811 06:17:47.109791 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd9f752fd5454ac8befd66f9cb3d6727" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea"
I20260811 06:17:47.109913 5370 raft_consensus.cc:3037] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.109951 4889 raft_consensus.cc:3037] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.112079 5370 raft_consensus.cc:2454] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 1.
I20260811 06:17:47.112186 4889 raft_consensus.cc:2454] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 1.
I20260811 06:17:47.112474 5004 leader_election.cc:302] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 4125d19587154305bd262883bf3aab48; no voters:
I20260811 06:17:47.112927 5510 raft_consensus.cc:2781] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:47.113385 5510 raft_consensus.cc:686] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 1 LEADER]: Becoming Leader. State: Replica: 4125d19587154305bd262883bf3aab48, State: Running, Role: LEADER
I20260811 06:17:47.113960 5510 consensus_queue.cc:237] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } }
I20260811 06:17:47.118433 4580 catalog_manager.cc:5495] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4125d19587154305bd262883bf3aab48 (127.22.121.2). New cstate: current_term: 1 leader_uuid: "4125d19587154305bd262883bf3aab48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } health_report { overall_health: UNKNOWN } } }
I20260811 06:17:47.213245 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.208091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52720 (local address 127.22.121.2:32843)
0811 06:17:47.208372 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.208378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.208397 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:17:47.208503 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.208509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.208641 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.208788 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:47.209135 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.209860 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.211144 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.211599 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.211714 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.212162 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.212675 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.212804 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.212907 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.212987 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:17:47.220945 4889 raft_consensus.cc:1261] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Refusing update from remote peer 4125d19587154305bd262883bf3aab48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.221012 5370 raft_consensus.cc:1261] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Refusing update from remote peer 4125d19587154305bd262883bf3aab48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.221832 5513 consensus_queue.cc:1035] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.222321 5510 consensus_queue.cc:1035] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.246698 4580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:47.248106 4580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:47.251504 4580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Sending DeleteTablet for 3 replicas of tablet cd9f752fd5454ac8befd66f9cb3d6727
I20260811 06:17:47.252569 4838 tablet_service.cc:1514] Processing DeleteTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:60342
I20260811 06:17:47.252502 5087 tablet_service.cc:1514] Processing DeleteTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:52712
I20260811 06:17:47.253381 5533 tablet_replica.cc:330] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: stopping tablet replica
I20260811 06:17:47.253417 5330 tablet_service.cc:1514] Processing DeleteTablet for tablet cd9f752fd5454ac8befd66f9cb3d6727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33856
I20260811 06:17:47.253814 5533 raft_consensus.cc:2229] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:47.254336 5533 raft_consensus.cc:2258] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.254577 4580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:47.255192 4580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:47.262629 4838 tablet_service.cc:1467] Processing CreateTablet for tablet c3c74f21f88e498e819729c2680e72de (DEFAULT_TABLE table=test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.262838 5329 tablet_service.cc:1467] Processing CreateTablet for tablet c3c74f21f88e498e819729c2680e72de (DEFAULT_TABLE table=test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.263075 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c74f21f88e498e819729c2680e72de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.263373 5329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c74f21f88e498e819729c2680e72de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.264349 5086 tablet_service.cc:1467] Processing CreateTablet for tablet c3c74f21f88e498e819729c2680e72de (DEFAULT_TABLE table=test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.264675 5533 ts_tablet_manager.cc:1905] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.264889 5086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c74f21f88e498e819729c2680e72de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.269834 5499 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Bootstrap starting.
I20260811 06:17:47.271472 5499 tablet_bootstrap.cc:654] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.276394 5533 ts_tablet_manager.cc:1918] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:47.276559 5533 log.cc:1196] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/cd9f752fd5454ac8befd66f9cb3d6727
I20260811 06:17:47.277070 5533 ts_tablet_manager.cc:1939] T cd9f752fd5454ac8befd66f9cb3d6727 P 9e8bf948919249aca93b16b7a4cdddea: Deleting consensus metadata
I20260811 06:17:47.278959 4555 catalog_manager.cc:4841] TS 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481): tablet cd9f752fd5454ac8befd66f9cb3d6727 (table test-table [id=9d0d8b6bc4994acebac25d6392926b91]) successfully deleted
I20260811 06:17:47.280652 5534 tablet_replica.cc:330] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: stopping tablet replica
I20260811 06:17:47.281117 5534 raft_consensus.cc:2229] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:47.281780 5534 raft_consensus.cc:2258] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.282166 5497 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Bootstrap starting.
I20260811 06:17:47.283479 5535 tablet_replica.cc:330] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: stopping tablet replica
I20260811 06:17:47.283778 5499 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: No bootstrap required, opened a new log
I20260811 06:17:47.283902 5535 raft_consensus.cc:2229] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:47.283948 5499 ts_tablet_manager.cc:1397] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.007s
I20260811 06:17:47.284126 5497 tablet_bootstrap.cc:654] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.284412 5535 raft_consensus.cc:2258] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.284901 5499 raft_consensus.cc:348] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.285166 5499 raft_consensus.cc:374] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.285308 5499 raft_consensus.cc:729] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.285594 5501 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Bootstrap starting.
I20260811 06:17:47.285792 5499 consensus_queue.cc:260] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.287084 5501 tablet_bootstrap.cc:654] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.288772 5499 ts_tablet_manager.cc:1428] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 06:17:47.293102 5535 ts_tablet_manager.cc:1905] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.294828 5534 ts_tablet_manager.cc:1905] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.297367 5501 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: No bootstrap required, opened a new log
I20260811 06:17:47.297605 5501 ts_tablet_manager.cc:1397] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20260811 06:17:47.299419 5501 raft_consensus.cc:348] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.299696 5501 raft_consensus.cc:374] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.299829 5501 raft_consensus.cc:729] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.300144 5501 consensus_queue.cc:260] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.300766 5501 ts_tablet_manager.cc:1428] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 06:17:47.300971 5535 ts_tablet_manager.cc:1918] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:47.301177 5535 log.cc:1196] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/cd9f752fd5454ac8befd66f9cb3d6727
I20260811 06:17:47.301775 5535 ts_tablet_manager.cc:1939] T cd9f752fd5454ac8befd66f9cb3d6727 P 2c019fd706b74eb083116074b119b8b9: Deleting consensus metadata
I20260811 06:17:47.303568 4556 catalog_manager.cc:4841] TS 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641): tablet cd9f752fd5454ac8befd66f9cb3d6727 (table test-table [id=9d0d8b6bc4994acebac25d6392926b91]) successfully deleted
I20260811 06:17:47.306344 5534 ts_tablet_manager.cc:1918] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:47.306542 5534 log.cc:1196] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/cd9f752fd5454ac8befd66f9cb3d6727
I20260811 06:17:47.307085 5534 ts_tablet_manager.cc:1939] T cd9f752fd5454ac8befd66f9cb3d6727 P 4125d19587154305bd262883bf3aab48: Deleting consensus metadata
I20260811 06:17:47.308374 5497 tablet_bootstrap.cc:492] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: No bootstrap required, opened a new log
I20260811 06:17:47.308539 5497 ts_tablet_manager.cc:1397] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.011s
I20260811 06:17:47.308835 4553 catalog_manager.cc:4841] TS 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843): tablet cd9f752fd5454ac8befd66f9cb3d6727 (table test-table [id=9d0d8b6bc4994acebac25d6392926b91]) successfully deleted
I20260811 06:17:47.309314 5497 raft_consensus.cc:348] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.309486 5497 raft_consensus.cc:374] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.309585 5497 raft_consensus.cc:729] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125d19587154305bd262883bf3aab48, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.309813 5497 consensus_queue.cc:260] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.310467 5497 ts_tablet_manager.cc:1428] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 06:17:47.351126 5513 raft_consensus.cc:482] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:47.351359 5513 raft_consensus.cc:504] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.352025 5513 leader_election.cc:288] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:47.352538 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c74f21f88e498e819729c2680e72de" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea" is_pre_election: true
I20260811 06:17:47.352643 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c74f21f88e498e819729c2680e72de" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9" is_pre_election: true
I20260811 06:17:47.352816 4889 raft_consensus.cc:2454] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 0.
I20260811 06:17:47.352881 5370 raft_consensus.cc:2454] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 0.
I20260811 06:17:47.353247 5003 leader_election.cc:302] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4125d19587154305bd262883bf3aab48, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:47.353585 5513 raft_consensus.cc:2781] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:47.353734 5513 raft_consensus.cc:482] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:47.353847 5513 raft_consensus.cc:3037] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.357016 5513 raft_consensus.cc:504] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.358110 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c74f21f88e498e819729c2680e72de" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea"
I20260811 06:17:47.358340 4889 raft_consensus.cc:3037] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.358426 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c74f21f88e498e819729c2680e72de" candidate_uuid: "4125d19587154305bd262883bf3aab48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9"
I20260811 06:17:47.358608 5370 raft_consensus.cc:3037] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.360679 5370 raft_consensus.cc:2454] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 1.
I20260811 06:17:47.361111 5513 leader_election.cc:288] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 election: Requested vote from peers 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:47.361135 5004 leader_election.cc:302] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 4125d19587154305bd262883bf3aab48; no voters:
I20260811 06:17:47.361258 4889 raft_consensus.cc:2454] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4125d19587154305bd262883bf3aab48 in term 1.
I20260811 06:17:47.361418 5513 raft_consensus.cc:2781] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:47.361725 5513 raft_consensus.cc:686] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 1 LEADER]: Becoming Leader. State: Replica: 4125d19587154305bd262883bf3aab48, State: Running, Role: LEADER
I20260811 06:17:47.362010 5513 consensus_queue.cc:237] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.364554 4573 catalog_manager.cc:5495] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4125d19587154305bd262883bf3aab48 (127.22.121.2). New cstate: current_term: 1 leader_uuid: "4125d19587154305bd262883bf3aab48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:17:47 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429063, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.62@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:47.418691 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.405575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35353 (local address 127.0.0.1:36320)
0811 06:17:47.406326 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.406341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.406471 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.406752 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.406761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.407498 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:47.407734 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.407744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.408807 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.408812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.409758 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.409769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.410446 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.410452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.410620 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.411196 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:47.411213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:47.413311 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:47.416208 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:47.416221 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:47.416229 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:47.416625 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:47.416944 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:47.416949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:47.416953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:47.417098 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:47.417594 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:47.417625 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:47.417791 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.417949 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.418211 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":474,"threads_started":1}
I20260811 06:17:47.418762 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.405502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.22.121.62:35353)
0811 06:17:47.406166 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.406176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.406420 (+ 244us) server_negotiation.cc:407] Connection header received
0811 06:17:47.406575 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.406580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.406631 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.406714 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:47.407897 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.408668 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.409886 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.410298 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.410404 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.413496 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:47.413501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:47.413504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:47.413531 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:47.416092 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:47.416731 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:47.416736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:47.416740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:47.416850 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:47.417199 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:47.417204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:47.417208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:47.417468 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:47.417594 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.418041 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.418339 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":493,"thread_start_us":339,"threads_started":1}
I20260811 06:17:47.424436 4573 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:36320:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:17:47.425652 4573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:47.432147 4838 tablet_service.cc:1467] Processing CreateTablet for tablet 43e2199a0c444dd5b319e91cae500b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1652af08e269467caae0727aa27ed737]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:17:47.432397 5086 tablet_service.cc:1467] Processing CreateTablet for tablet 43e2199a0c444dd5b319e91cae500b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1652af08e269467caae0727aa27ed737]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:17:47.432729 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e2199a0c444dd5b319e91cae500b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.432940 5086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e2199a0c444dd5b319e91cae500b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.435765 5330 tablet_service.cc:1467] Processing CreateTablet for tablet 43e2199a0c444dd5b319e91cae500b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1652af08e269467caae0727aa27ed737]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:17:47.436280 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e2199a0c444dd5b319e91cae500b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.438318 5501 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: Bootstrap starting.
I20260811 06:17:47.439934 5501 tablet_bootstrap.cc:654] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.441097 5497 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: Bootstrap starting.
I20260811 06:17:47.442895 5497 tablet_bootstrap.cc:654] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.442981 5499 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: Bootstrap starting.
I20260811 06:17:47.444479 5499 tablet_bootstrap.cc:654] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.450875 5499 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: No bootstrap required, opened a new log
I20260811 06:17:47.451076 5499 ts_tablet_manager.cc:1397] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260811 06:17:47.451574 5501 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: No bootstrap required, opened a new log
I20260811 06:17:47.451741 5501 ts_tablet_manager.cc:1397] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260811 06:17:47.451980 5499 raft_consensus.cc:348] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.452263 5499 raft_consensus.cc:374] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.452385 5499 raft_consensus.cc:729] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.452427 5501 raft_consensus.cc:348] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.452636 5501 raft_consensus.cc:374] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.452715 5501 raft_consensus.cc:729] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.452867 5499 consensus_queue.cc:260] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.452962 5501 consensus_queue.cc:260] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.453387 5509 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.453387 5506 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.453698 5509 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.453733 5506 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.454144 5501 ts_tablet_manager.cc:1428] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:47.455634 5497 tablet_bootstrap.cc:492] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: No bootstrap required, opened a new log
I20260811 06:17:47.455839 5497 ts_tablet_manager.cc:1397] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260811 06:17:47.456061 5499 ts_tablet_manager.cc:1428] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 06:17:47.456595 5497 raft_consensus.cc:348] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.456907 5497 raft_consensus.cc:374] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.457046 5497 raft_consensus.cc:729] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125d19587154305bd262883bf3aab48, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.457337 5497 consensus_queue.cc:260] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.458009 5497 ts_tablet_manager.cc:1428] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:47.457865 5513 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.458225 5513 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.485535 5506 raft_consensus.cc:482] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:47.485695 5506 raft_consensus.cc:504] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.487087 5506 leader_election.cc:288] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481), 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843)
I20260811 06:17:47.496654 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.487620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37131 (local address 127.22.121.1:36481)
0811 06:17:47.487804 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.487811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.488724 (+ 913us) server_negotiation.cc:407] Connection header received
0811 06:17:47.489006 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.489011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.489083 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.489169 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.489970 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.491129 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.493934 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.495193 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.495365 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.495439 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.496116 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.496327 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:17:47.497419 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.487397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36481 (local address 127.22.121.3:37131)
0811 06:17:47.488609 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.488630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.488830 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.489332 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.489335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.489355 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.489787 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.489800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.491389 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.491395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.493713 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.493729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.495680 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.495687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.495913 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.495948 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.496859 (+ 911us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.496995 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":27,"thread_start_us":926,"threads_started":1}
I20260811 06:17:47.498602 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.487517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32843 (local address 127.22.121.3:58101)
0811 06:17:47.488889 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.488904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.489069 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.489464 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.489468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.489493 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.489909 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.489919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.491205 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.491211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.493928 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.493974 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.497840 (+ 3866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.497847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.498018 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.498050 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.498144 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.498257 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":88,"thread_start_us":1103,"threads_started":1}
I20260811 06:17:47.499085 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.487991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58101 (local address 127.22.121.2:32843)
0811 06:17:47.488176 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.488182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.488982 (+ 800us) server_negotiation.cc:407] Connection header received
0811 06:17:47.489242 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.489247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.489328 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.489427 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.490068 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.491042 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.494186 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.496123 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.497382 (+ 1259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.497636 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.498235 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.498662 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:17:47.503942 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e2199a0c444dd5b319e91cae500b43" candidate_uuid: "2c019fd706b74eb083116074b119b8b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48" is_pre_election: true
I20260811 06:17:47.504386 5133 raft_consensus.cc:2454] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c019fd706b74eb083116074b119b8b9 in term 0.
I20260811 06:17:47.505064 5236 leader_election.cc:302] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 4125d19587154305bd262883bf3aab48; no voters:
I20260811 06:17:47.506168 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e2199a0c444dd5b319e91cae500b43" candidate_uuid: "2c019fd706b74eb083116074b119b8b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea" is_pre_election: true
I20260811 06:17:47.506390 4889 raft_consensus.cc:2454] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c019fd706b74eb083116074b119b8b9 in term 0.
I20260811 06:17:47.506692 5506 raft_consensus.cc:2781] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:47.506798 5506 raft_consensus.cc:482] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:47.506877 5506 raft_consensus.cc:3037] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.509353 5506 raft_consensus.cc:504] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.509871 5506 leader_election.cc:288] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [CANDIDATE]: Term 1 election: Requested vote from peers 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481), 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843)
I20260811 06:17:47.510324 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e2199a0c444dd5b319e91cae500b43" candidate_uuid: "2c019fd706b74eb083116074b119b8b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e8bf948919249aca93b16b7a4cdddea"
I20260811 06:17:47.510406 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e2199a0c444dd5b319e91cae500b43" candidate_uuid: "2c019fd706b74eb083116074b119b8b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48"
I20260811 06:17:47.510532 4889 raft_consensus.cc:3037] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.510614 5133 raft_consensus.cc:3037] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.512458 4889 raft_consensus.cc:2454] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c019fd706b74eb083116074b119b8b9 in term 1.
I20260811 06:17:47.512971 5237 leader_election.cc:302] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:47.513296 5506 raft_consensus.cc:2781] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:47.513748 5506 raft_consensus.cc:686] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Running, Role: LEADER
I20260811 06:17:47.513829 5133 raft_consensus.cc:2454] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c019fd706b74eb083116074b119b8b9 in term 1.
I20260811 06:17:47.514185 5506 consensus_queue.cc:237] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.516477 5566 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: TxnStatusTablet state changed. Reason: New leader 2c019fd706b74eb083116074b119b8b9. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.516850 5566 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:17:47.517475 5571 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:17:47.517666 5571 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:17:47.517740 4580 catalog_manager.cc:5495] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c019fd706b74eb083116074b119b8b9 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } health_report { overall_health: HEALTHY } } }
I20260811 06:17:47.546598 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.535000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.0.0.1:33872)
0811 06:17:47.535241 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.535283 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.535436 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.536045 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.536049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.536058 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:47.536190 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.536197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.537231 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.537236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.538241 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.538253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.538948 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.538954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.539104 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.539157 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.546011 (+ 6854us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.546059 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.546140 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.546297 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:17:47.547295 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.534948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.22.121.3:34641)
0811 06:17:47.535112 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.535118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.535345 (+ 227us) server_negotiation.cc:407] Connection header received
0811 06:17:47.535629 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.535633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.535724 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.535869 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:47.536328 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.537113 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.538371 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.538808 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.538922 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.539346 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.539946 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.540089 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.546891 (+ 6802us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.546974 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:17:47.551752 4889 raft_consensus.cc:1261] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Refusing update from remote peer 2c019fd706b74eb083116074b119b8b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.552492 5506 consensus_queue.cc:1035] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.554196 5133 raft_consensus.cc:1261] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Refusing update from remote peer 2c019fd706b74eb083116074b119b8b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.554940 5506 consensus_queue.cc:1035] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.568513 5509 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: TxnStatusTablet state changed. Reason: New leader 2c019fd706b74eb083116074b119b8b9. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.568789 5509 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.571846 5506 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.572161 5506 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:17:47.572342 5520 mvcc.cc:205] Tried to move back new op lower bound from 7317213460684615680 to 7317213460541820928. Current Snapshot: MvccSnapshot[applied={T|T < 7317213460684615680}]
I20260811 06:17:47.572795 5519 mvcc.cc:205] Tried to move back new op lower bound from 7317213460684615680 to 7317213460541820928. Current Snapshot: MvccSnapshot[applied={T|T < 7317213460684615680 or (T in {7317213460684615680})}]
I20260811 06:17:47.573963 5509 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.574154 5509 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 9e8bf948919249aca93b16b7a4cdddea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.581753 5513 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: TxnStatusTablet state changed. Reason: New leader 2c019fd706b74eb083116074b119b8b9. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.582077 5513 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.584722 5566 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.584944 5566 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 2c019fd706b74eb083116074b119b8b9: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:17:47.588068 5513 tablet_replica.cc:439] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c019fd706b74eb083116074b119b8b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } } }
I20260811 06:17:47.588317 5513 tablet_replica.cc:442] T 43e2199a0c444dd5b319e91cae500b43 P 4125d19587154305bd262883bf3aab48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:17:47.616626 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.609373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:59471 (local address 127.22.121.3:34641)
0811 06:17:47.609554 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.609560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.610104 (+ 544us) server_negotiation.cc:407] Connection header received
0811 06:17:47.610300 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.610306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.610419 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.610520 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:47.611063 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.612252 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.614304 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.615062 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.615181 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.615753 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.615895 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.615996 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.616239 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.616345 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:17:47.616688 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.609368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.22.121.2:59471)
0811 06:17:47.609986 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.610006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.610163 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.610561 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.610566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.610580 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:47.610863 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.610880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.612416 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.612421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.614157 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.614173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.615255 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.615281 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.615479 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.615549 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.616016 (+ 467us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.616030 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.616139 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.616293 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
I20260811 06:17:47.630226 5370 raft_consensus.cc:1261] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Refusing update from remote peer 4125d19587154305bd262883bf3aab48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.630946 5525 consensus_queue.cc:1035] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.634009 4889 raft_consensus.cc:1261] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Refusing update from remote peer 4125d19587154305bd262883bf3aab48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.634639 5525 consensus_queue.cc:1035] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.640666 5518 mvcc.cc:205] Tried to move back new op lower bound from 7317213461006589952 to 7317213459917086720. Current Snapshot: MvccSnapshot[applied={T|T < 7317213461006589952}]
I20260811 06:17:47.690492 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.674449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:59289 (local address 127.22.121.2:32843)
0811 06:17:47.674934 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.674940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.683503 (+ 8563us) server_negotiation.cc:407] Connection header received
0811 06:17:47.683934 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.683940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.684030 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.684092 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:47.684937 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.686166 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.688358 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.689072 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.689199 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.689705 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.689831 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.689927 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.690073 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.690157 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260811 06:17:47.690603 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.674019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32843 (local address 127.22.121.3:59289)
0811 06:17:47.683404 (+ 9385us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.683424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.683740 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.684292 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.684297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.684310 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:47.684712 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.684730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.686333 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.686338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.688199 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.688217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.689264 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.689271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.689448 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.689522 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.689942 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.689962 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.690048 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.690202 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9139,"thread_start_us":9494,"threads_started":1}
I20260811 06:17:47.738232 4580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:47.738535 4580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:47.741847 4580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Sending DeleteTablet for 3 replicas of tablet c3c74f21f88e498e819729c2680e72de
I20260811 06:17:47.742590 4838 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c74f21f88e498e819729c2680e72de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:60342
I20260811 06:17:47.742878 5533 tablet_replica.cc:330] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: stopping tablet replica
I20260811 06:17:47.743001 5086 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c74f21f88e498e819729c2680e72de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:52712
I20260811 06:17:47.743088 5533 raft_consensus.cc:2229] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:47.743251 5534 tablet_replica.cc:330] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: stopping tablet replica
I20260811 06:17:47.743299 5533 raft_consensus.cc:2258] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.743350 5330 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c74f21f88e498e819729c2680e72de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:47 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33856
I20260811 06:17:47.743465 5534 raft_consensus.cc:2229] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:47.743613 5535 tablet_replica.cc:330] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: stopping tablet replica
I20260811 06:17:47.743775 5534 raft_consensus.cc:2258] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.743805 5535 raft_consensus.cc:2229] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:47.743978 5535 raft_consensus.cc:2258] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:47.744544 5533 ts_tablet_manager.cc:1905] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.745128 5534 ts_tablet_manager.cc:1905] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.751396 5533 ts_tablet_manager.cc:1918] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:47.751590 5533 log.cc:1196] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/c3c74f21f88e498e819729c2680e72de
I20260811 06:17:47.752036 5533 ts_tablet_manager.cc:1939] T c3c74f21f88e498e819729c2680e72de P 9e8bf948919249aca93b16b7a4cdddea: Deleting consensus metadata
I20260811 06:17:47.752328 5535 ts_tablet_manager.cc:1905] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:47.753150 4555 catalog_manager.cc:4841] TS 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481): tablet c3c74f21f88e498e819729c2680e72de (table test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]) successfully deleted
I20260811 06:17:47.761530 5535 ts_tablet_manager.cc:1918] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:47.761662 5535 log.cc:1196] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/c3c74f21f88e498e819729c2680e72de
I20260811 06:17:47.762017 5534 ts_tablet_manager.cc:1918] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:47.762185 5535 ts_tablet_manager.cc:1939] T c3c74f21f88e498e819729c2680e72de P 2c019fd706b74eb083116074b119b8b9: Deleting consensus metadata
I20260811 06:17:47.762212 5534 log.cc:1196] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/c3c74f21f88e498e819729c2680e72de
I20260811 06:17:47.762624 5534 ts_tablet_manager.cc:1939] T c3c74f21f88e498e819729c2680e72de P 4125d19587154305bd262883bf3aab48: Deleting consensus metadata
I20260811 06:17:47.763302 4556 catalog_manager.cc:4841] TS 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641): tablet c3c74f21f88e498e819729c2680e72de (table test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]) successfully deleted
I20260811 06:17:47.763651 4553 catalog_manager.cc:4841] TS 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843): tablet c3c74f21f88e498e819729c2680e72de (table test-table [id=0066f3bdd6854fd5b113a3a10f86c31b]) successfully deleted
Aug 11 06:17:47 dist-test-slave-gjrz krb5kdc[4494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:17:47 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:47 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:17:47.812512 5553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:47.800588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.22.121.62:35353)
0811 06:17:47.800764 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.800769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.801334 (+ 565us) server_negotiation.cc:407] Connection header received
0811 06:17:47.801512 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.801516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.801582 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.801662 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:47.803416 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.804170 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.805975 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.806403 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.806496 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.812269 (+ 5773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36330: BlockingRecv error: recv got EOF from 127.0.0.1:36330 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:17:47 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM
I20260811 06:17:47.829368 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.815064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.22.121.62:35353)
0811 06:17:47.815220 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.815225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.815515 (+ 290us) server_negotiation.cc:407] Connection header received
0811 06:17:47.815636 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.815642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.815712 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.815808 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:47.816913 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.817949 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.819200 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.819884 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.820017 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.823658 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:47.823666 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:47.823670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:47.823710 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:47.826764 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:47.827369 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:47.827375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:47.827379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:47.827543 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:47.827850 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:47.827854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:47.827858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:47.828237 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:47.828493 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.828783 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.829018 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:17:47.832669 4580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:47.833412 4580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:47.840101 4838 tablet_service.cc:1467] Processing CreateTablet for tablet cabbc768b28941fdaa17a630661c03b9 (DEFAULT_TABLE table=test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.840569 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabbc768b28941fdaa17a630661c03b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.840505 5086 tablet_service.cc:1467] Processing CreateTablet for tablet cabbc768b28941fdaa17a630661c03b9 (DEFAULT_TABLE table=test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.840941 5086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabbc768b28941fdaa17a630661c03b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.843842 5330 tablet_service.cc:1467] Processing CreateTablet for tablet cabbc768b28941fdaa17a630661c03b9 (DEFAULT_TABLE table=test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:47.844270 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabbc768b28941fdaa17a630661c03b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:47.846058 5501 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Bootstrap starting.
I20260811 06:17:47.847198 5497 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Bootstrap starting.
I20260811 06:17:47.847692 5501 tablet_bootstrap.cc:654] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.848688 5497 tablet_bootstrap.cc:654] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.852625 5499 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Bootstrap starting.
I20260811 06:17:47.854409 5499 tablet_bootstrap.cc:654] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:47.858404 5497 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: No bootstrap required, opened a new log
I20260811 06:17:47.858554 5497 ts_tablet_manager.cc:1397] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 06:17:47.858645 5499 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: No bootstrap required, opened a new log
I20260811 06:17:47.858788 5499 ts_tablet_manager.cc:1397] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:17:47.859344 5497 raft_consensus.cc:348] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.859598 5497 raft_consensus.cc:374] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.859558 5499 raft_consensus.cc:348] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.859735 5497 raft_consensus.cc:729] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125d19587154305bd262883bf3aab48, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.859858 5499 raft_consensus.cc:374] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.859965 5499 raft_consensus.cc:729] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.859997 5497 consensus_queue.cc:260] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.860193 5499 consensus_queue.cc:260] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.860714 5499 ts_tablet_manager.cc:1428] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:47.861997 5497 ts_tablet_manager.cc:1428] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:47.863013 5501 tablet_bootstrap.cc:492] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: No bootstrap required, opened a new log
I20260811 06:17:47.863229 5501 ts_tablet_manager.cc:1397] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260811 06:17:47.864051 5501 raft_consensus.cc:348] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.864362 5501 raft_consensus.cc:374] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:47.864481 5501 raft_consensus.cc:729] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Initialized, Role: FOLLOWER
I20260811 06:17:47.864770 5501 consensus_queue.cc:260] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.865396 5501 ts_tablet_manager.cc:1428] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:47.932019 5509 raft_consensus.cc:482] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:47.932241 5509 raft_consensus.cc:504] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.934126 5509 leader_election.cc:288] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:47.942188 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.934786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:40279 (local address 127.22.121.2:32843)
0811 06:17:47.934996 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.935002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.936357 (+ 1355us) server_negotiation.cc:407] Connection header received
0811 06:17:47.936539 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.936545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.936630 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.936727 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.937349 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.938263 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.940083 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.941181 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.941334 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.941407 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.941765 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.941905 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:17:47.942291 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.934568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:32843 (local address 127.22.121.1:40279)
0811 06:17:47.936237 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.936254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.936400 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.936766 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.936771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.936796 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.937183 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.937195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.938386 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.938391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.939932 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.939945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.941409 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.941415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.941580 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.941616 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.941713 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.941848 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1546,"mutex_wait_us":411,"thread_start_us":992,"threads_started":1}
I20260811 06:17:47.942983 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.934757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:37867 (local address 127.22.121.3:34641)
0811 06:17:47.934956 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.934963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.935706 (+ 743us) server_negotiation.cc:407] Connection header received
0811 06:17:47.935869 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.935875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.935959 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.936050 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:17:47.936810 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.937827 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.940398 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.941676 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.941841 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.941918 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.942622 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.942722 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:17:47.942998 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabbc768b28941fdaa17a630661c03b9" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48" is_pre_election: true
I20260811 06:17:47.943229 5133 raft_consensus.cc:2454] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 0.
I20260811 06:17:47.943800 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.934551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.22.121.1:37867)
0811 06:17:47.935578 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:47.935598 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:47.935740 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:47.936201 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:47.936206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:47.936236 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:17:47.936602 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.936616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.937958 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.937971 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.940219 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:47.940234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.942089 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.942095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:47.942317 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.942348 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:17:47.943234 (+ 886us) client_negotiation.cc:240] Negotiation successful
0811 06:17:47.943382 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":27,"spinlock_wait_cycles":7936,"thread_start_us":694,"threads_started":1}
I20260811 06:17:47.943765 4746 leader_election.cc:302] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4125d19587154305bd262883bf3aab48, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:47.944273 5509 raft_consensus.cc:2781] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:47.944396 5509 raft_consensus.cc:482] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:47.944491 5509 raft_consensus.cc:3037] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.944581 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabbc768b28941fdaa17a630661c03b9" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9" is_pre_election: true
I20260811 06:17:47.944805 5370 raft_consensus.cc:2454] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 0.
I20260811 06:17:47.947510 5509 raft_consensus.cc:504] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.948138 5509 leader_election.cc:288] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 election: Requested vote from peers 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:47.948540 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabbc768b28941fdaa17a630661c03b9" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48"
I20260811 06:17:47.948751 5133 raft_consensus.cc:3037] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.948797 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabbc768b28941fdaa17a630661c03b9" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9"
I20260811 06:17:47.948992 5370 raft_consensus.cc:3037] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:47.951625 5133 raft_consensus.cc:2454] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 1.
I20260811 06:17:47.951931 5370 raft_consensus.cc:2454] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 1.
I20260811 06:17:47.952013 4746 leader_election.cc:302] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4125d19587154305bd262883bf3aab48, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:47.952263 5509 raft_consensus.cc:2781] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:47.952654 5509 raft_consensus.cc:686] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 1 LEADER]: Becoming Leader. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Running, Role: LEADER
I20260811 06:17:47.953045 5509 consensus_queue.cc:237] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:47.956470 4580 catalog_manager.cc:5495] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1). New cstate: current_term: 1 leader_uuid: "9e8bf948919249aca93b16b7a4cdddea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } health_report { overall_health: UNKNOWN } } }
I20260811 06:17:47.982473 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:47.977173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.22.121.1:36481)
0811 06:17:47.977328 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:47.977334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:47.977509 (+ 175us) server_negotiation.cc:407] Connection header received
0811 06:17:47.977748 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:47.977752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:47.977828 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:47.977964 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:47.978324 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.979048 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.980407 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:47.980812 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:47.980929 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:47.981354 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:47.981873 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:47.982034 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:47.982123 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 06:17:47.982203 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:17:47.986241 5370 raft_consensus.cc:1261] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Refusing update from remote peer 9e8bf948919249aca93b16b7a4cdddea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.986375 5133 raft_consensus.cc:1261] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Refusing update from remote peer 9e8bf948919249aca93b16b7a4cdddea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:47.986996 5624 consensus_queue.cc:1035] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Connected to new peer: Peer: permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:47.987396 5509 consensus_queue.cc:1035] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:48.003854 4573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:48.004143 4573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:48.007167 4573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Sending DeleteTablet for 3 replicas of tablet cabbc768b28941fdaa17a630661c03b9
I20260811 06:17:48.007997 4838 tablet_service.cc:1514] Processing DeleteTablet for tablet cabbc768b28941fdaa17a630661c03b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:60342
I20260811 06:17:48.007978 5086 tablet_service.cc:1514] Processing DeleteTablet for tablet cabbc768b28941fdaa17a630661c03b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:52712
I20260811 06:17:48.008045 5330 tablet_service.cc:1514] Processing DeleteTablet for tablet cabbc768b28941fdaa17a630661c03b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33856
I20260811 06:17:48.008316 5535 tablet_replica.cc:330] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: stopping tablet replica
I20260811 06:17:48.008327 5533 tablet_replica.cc:330] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: stopping tablet replica
I20260811 06:17:48.008494 5535 raft_consensus.cc:2229] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:48.008513 5533 raft_consensus.cc:2229] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:48.008716 5535 raft_consensus.cc:2258] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.009009 5533 raft_consensus.cc:2258] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.009303 5534 tablet_replica.cc:330] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: stopping tablet replica
I20260811 06:17:48.009487 5534 raft_consensus.cc:2229] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:48.009614 5535 ts_tablet_manager.cc:1905] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.009714 5534 raft_consensus.cc:2258] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.011369 5534 ts_tablet_manager.cc:1905] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.014357 5535 ts_tablet_manager.cc:1918] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:48.014472 5535 log.cc:1196] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/cabbc768b28941fdaa17a630661c03b9
I20260811 06:17:48.014818 5535 ts_tablet_manager.cc:1939] T cabbc768b28941fdaa17a630661c03b9 P 2c019fd706b74eb083116074b119b8b9: Deleting consensus metadata
I20260811 06:17:48.015149 4574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:17:48.015874 4574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:17:48.016019 4556 catalog_manager.cc:4841] TS 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641): tablet cabbc768b28941fdaa17a630661c03b9 (table test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]) successfully deleted
I20260811 06:17:48.017666 5533 ts_tablet_manager.cc:1905] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.023684 5534 ts_tablet_manager.cc:1918] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:48.023890 5534 log.cc:1196] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/cabbc768b28941fdaa17a630661c03b9
I20260811 06:17:48.024371 5534 ts_tablet_manager.cc:1939] T cabbc768b28941fdaa17a630661c03b9 P 4125d19587154305bd262883bf3aab48: Deleting consensus metadata
I20260811 06:17:48.025604 5330 tablet_service.cc:1467] Processing CreateTablet for tablet d76b32b56362483795f60bae9f402970 (DEFAULT_TABLE table=test-table [id=1b463e15a94e480f96c790c632b24c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:48.025950 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76b32b56362483795f60bae9f402970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:48.027555 5533 ts_tablet_manager.cc:1918] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:17:48.027691 5533 log.cc:1196] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/cabbc768b28941fdaa17a630661c03b9
I20260811 06:17:48.028146 5533 ts_tablet_manager.cc:1939] T cabbc768b28941fdaa17a630661c03b9 P 9e8bf948919249aca93b16b7a4cdddea: Deleting consensus metadata
I20260811 06:17:48.030652 5499 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Bootstrap starting.
I20260811 06:17:48.031032 5086 tablet_service.cc:1467] Processing CreateTablet for tablet d76b32b56362483795f60bae9f402970 (DEFAULT_TABLE table=test-table [id=1b463e15a94e480f96c790c632b24c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:48.031553 4553 catalog_manager.cc:4841] TS 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843): tablet cabbc768b28941fdaa17a630661c03b9 (table test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]) successfully deleted
I20260811 06:17:48.031952 4555 catalog_manager.cc:4841] TS 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481): tablet cabbc768b28941fdaa17a630661c03b9 (table test-table [id=3ec42d7d31d74ef2982acbfdaa077b17]) successfully deleted
I20260811 06:17:48.032233 5499 tablet_bootstrap.cc:654] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:48.032259 5086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76b32b56362483795f60bae9f402970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:48.037204 4838 tablet_service.cc:1467] Processing CreateTablet for tablet d76b32b56362483795f60bae9f402970 (DEFAULT_TABLE table=test-table [id=1b463e15a94e480f96c790c632b24c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:17:48.037652 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76b32b56362483795f60bae9f402970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:48.047438 5501 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Bootstrap starting.
I20260811 06:17:48.049055 5501 tablet_bootstrap.cc:654] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:48.049993 5497 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Bootstrap starting.
I20260811 06:17:48.051163 5497 tablet_bootstrap.cc:654] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:48.053380 5501 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: No bootstrap required, opened a new log
I20260811 06:17:48.053586 5501 ts_tablet_manager.cc:1397] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:17:48.054203 5501 raft_consensus.cc:348] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.054411 5501 raft_consensus.cc:374] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:48.054497 5501 raft_consensus.cc:729] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Initialized, Role: FOLLOWER
I20260811 06:17:48.054756 5501 consensus_queue.cc:260] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.055239 5501 ts_tablet_manager.cc:1428] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:17:48.056022 5499 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: No bootstrap required, opened a new log
I20260811 06:17:48.056187 5499 ts_tablet_manager.cc:1397] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.010s
I20260811 06:17:48.057080 5497 tablet_bootstrap.cc:492] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: No bootstrap required, opened a new log
I20260811 06:17:48.057246 5497 ts_tablet_manager.cc:1397] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 06:17:48.057998 5497 raft_consensus.cc:348] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.058260 5497 raft_consensus.cc:374] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:48.058347 5497 raft_consensus.cc:729] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125d19587154305bd262883bf3aab48, State: Initialized, Role: FOLLOWER
I20260811 06:17:48.058575 5497 consensus_queue.cc:260] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.059001 5497 ts_tablet_manager.cc:1428] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:48.059787 5499 raft_consensus.cc:348] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.060061 5499 raft_consensus.cc:374] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:48.060186 5499 raft_consensus.cc:729] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c019fd706b74eb083116074b119b8b9, State: Initialized, Role: FOLLOWER
I20260811 06:17:48.060422 5499 consensus_queue.cc:260] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.060953 5499 ts_tablet_manager.cc:1428] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 06:17:48.112222 5509 raft_consensus.cc:482] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:17:48.112432 5509 raft_consensus.cc:504] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.113158 5509 leader_election.cc:288] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:48.113628 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76b32b56362483795f60bae9f402970" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48" is_pre_election: true
I20260811 06:17:48.113673 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76b32b56362483795f60bae9f402970" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9" is_pre_election: true
I20260811 06:17:48.113866 5133 raft_consensus.cc:2454] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 0.
I20260811 06:17:48.113883 5370 raft_consensus.cc:2454] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 0.
I20260811 06:17:48.114321 4746 leader_election.cc:302] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4125d19587154305bd262883bf3aab48, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:48.114624 5509 raft_consensus.cc:2781] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:17:48.114764 5509 raft_consensus.cc:482] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:17:48.114861 5509 raft_consensus.cc:3037] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:48.117111 5509 raft_consensus.cc:504] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.117640 5509 leader_election.cc:288] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 election: Requested vote from peers 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843), 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641)
I20260811 06:17:48.118110 5370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76b32b56362483795f60bae9f402970" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c019fd706b74eb083116074b119b8b9"
I20260811 06:17:48.118134 5133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76b32b56362483795f60bae9f402970" candidate_uuid: "9e8bf948919249aca93b16b7a4cdddea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4125d19587154305bd262883bf3aab48"
I20260811 06:17:48.118320 5133 raft_consensus.cc:3037] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:48.118304 5370 raft_consensus.cc:3037] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:48.120424 5370 raft_consensus.cc:2454] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 1.
I20260811 06:17:48.120924 4748 leader_election.cc:302] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c019fd706b74eb083116074b119b8b9, 9e8bf948919249aca93b16b7a4cdddea; no voters:
I20260811 06:17:48.121246 5133 raft_consensus.cc:2454] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8bf948919249aca93b16b7a4cdddea in term 1.
I20260811 06:17:48.121704 5509 raft_consensus.cc:2781] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:48.122241 5509 raft_consensus.cc:686] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 1 LEADER]: Becoming Leader. State: Replica: 9e8bf948919249aca93b16b7a4cdddea, State: Running, Role: LEADER
I20260811 06:17:48.122713 5509 consensus_queue.cc:237] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } }
I20260811 06:17:48.125602 4574 catalog_manager.cc:5495] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1). New cstate: current_term: 1 leader_uuid: "9e8bf948919249aca93b16b7a4cdddea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e8bf948919249aca93b16b7a4cdddea" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 } health_report { overall_health: UNKNOWN } } }
I20260811 06:17:48.177294 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.171291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:57013 (local address 127.22.121.3:34641)
0811 06:17:48.171451 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:48.171458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:48.172150 (+ 692us) server_negotiation.cc:407] Connection header received
0811 06:17:48.172306 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:48.172311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:48.172395 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:48.172501 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:48.172962 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.174104 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.175551 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.176152 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.176277 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.176644 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:48.176775 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:48.176862 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:48.176969 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 06:17:48.177046 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 06:17:48.177322 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.171353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:34641 (local address 127.22.121.1:57013)
0811 06:17:48.172041 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:48.172059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:48.172191 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:48.172494 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:48.172497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:48.172506 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:48.172826 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:48.172834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.174225 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.174229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:48.175423 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:48.175436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.176309 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.176314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:48.176442 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.176492 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:48.176864 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:48.176872 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:17:48.176930 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 06:17:48.177029 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":378,"threads_started":1}
I20260811 06:17:48.185356 5370 raft_consensus.cc:1261] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Refusing update from remote peer 9e8bf948919249aca93b16b7a4cdddea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:48.185449 5127 raft_consensus.cc:1261] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Refusing update from remote peer 9e8bf948919249aca93b16b7a4cdddea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:17:48.186017 5624 consensus_queue.cc:1035] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Connected to new peer: Peer: permanent_uuid: "4125d19587154305bd262883bf3aab48" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:48.186468 5509 consensus_queue.cc:1035] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c019fd706b74eb083116074b119b8b9" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:17:48.216747 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.209991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:36481 (local address 127.22.121.3:57633)
0811 06:17:48.210540 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:48.210565 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:48.210693 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:48.210962 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:48.210966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:48.210978 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:17:48.211195 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:48.211206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.212384 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.212390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:48.214720 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:48.214735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.215600 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.215607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:48.215743 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.215779 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:17:48.216227 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:17:48.216236 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:17:48.216301 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 06:17:48.216434 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":332,"threads_started":1}
I20260811 06:17:48.216744 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.210425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:57633 (local address 127.22.121.1:36481)
0811 06:17:48.210568 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:48.210573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:48.210620 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:17:48.210804 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:48.210808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:48.210867 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:48.210936 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:17:48.211340 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.212249 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.214850 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.215440 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.215561 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.215927 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:17:48.216090 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:17:48.216170 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:48.216347 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:17:48.216422 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 06:17:48.243819 4573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:48.244025 4573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:17:48.246691 4573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce66c752a039426eb2002dafc5f9a53d: Sending DeleteTablet for 3 replicas of tablet d76b32b56362483795f60bae9f402970
I20260811 06:17:48.247494 4838 tablet_service.cc:1514] Processing DeleteTablet for tablet d76b32b56362483795f60bae9f402970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:60342
I20260811 06:17:48.247663 5086 tablet_service.cc:1514] Processing DeleteTablet for tablet d76b32b56362483795f60bae9f402970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:52712
I20260811 06:17:48.247777 5533 tablet_replica.cc:330] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: stopping tablet replica
I20260811 06:17:48.247917 5534 tablet_replica.cc:330] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: stopping tablet replica
I20260811 06:17:48.247977 5533 raft_consensus.cc:2229] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:17:48.248117 5534 raft_consensus.cc:2229] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:48.248270 5533 raft_consensus.cc:2258] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.248287 5534 raft_consensus.cc:2258] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.249202 5330 tablet_service.cc:1514] Processing DeleteTablet for tablet d76b32b56362483795f60bae9f402970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:17:48 UTC) from {username='oryx', principal='oryx/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33856
I20260811 06:17:48.249303 5534 ts_tablet_manager.cc:1905] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.249449 5535 tablet_replica.cc:330] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: stopping tablet replica
I20260811 06:17:48.249475 5533 ts_tablet_manager.cc:1905] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.249635 5535 raft_consensus.cc:2229] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:17:48.249799 5535 raft_consensus.cc:2258] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:17:48.250340 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 4718
I20260811 06:17:48.253894 5535 ts_tablet_manager.cc:1905] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:17:48.256062 5534 ts_tablet_manager.cc:1918] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:48.256254 5533 ts_tablet_manager.cc:1918] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:48.256291 5534 log.cc:1196] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/d76b32b56362483795f60bae9f402970
I20260811 06:17:48.256408 5533 log.cc:1196] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/d76b32b56362483795f60bae9f402970
I20260811 06:17:48.256830 5533 ts_tablet_manager.cc:1939] T d76b32b56362483795f60bae9f402970 P 9e8bf948919249aca93b16b7a4cdddea: Deleting consensus metadata
I20260811 06:17:48.256959 5534 ts_tablet_manager.cc:1939] T d76b32b56362483795f60bae9f402970 P 4125d19587154305bd262883bf3aab48: Deleting consensus metadata
I20260811 06:17:48.257992 4553 catalog_manager.cc:4841] TS 4125d19587154305bd262883bf3aab48 (127.22.121.2:32843): tablet d76b32b56362483795f60bae9f402970 (table test-table [id=1b463e15a94e480f96c790c632b24c7f]) successfully deleted
I20260811 06:17:48.258435 4555 catalog_manager.cc:4841] TS 9e8bf948919249aca93b16b7a4cdddea (127.22.121.1:36481): tablet d76b32b56362483795f60bae9f402970 (table test-table [id=1b463e15a94e480f96c790c632b24c7f]) successfully deleted
I20260811 06:17:48.261107 5535 ts_tablet_manager.cc:1918] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:17:48.261243 5535 log.cc:1196] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/d76b32b56362483795f60bae9f402970
I20260811 06:17:48.261559 5535 ts_tablet_manager.cc:1939] T d76b32b56362483795f60bae9f402970 P 2c019fd706b74eb083116074b119b8b9: Deleting consensus metadata
I20260811 06:17:48.262413 4556 catalog_manager.cc:4841] TS 2c019fd706b74eb083116074b119b8b9 (127.22.121.3:34641): tablet d76b32b56362483795f60bae9f402970 (table test-table [id=1b463e15a94e480f96c790c632b24c7f]) successfully deleted
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.1@KRBTEST.COM
I20260811 06:17:48.274385 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.257185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.22.121.1:36481)
0811 06:17:48.257323 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:48.257330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:48.258415 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 06:17:48.258709 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:48.258715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:48.258793 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:48.258866 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:48.260716 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.261852 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.263515 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.264185 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.264312 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.267960 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:48.267972 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:48.267989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:48.268029 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:48.271745 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.272377 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.272387 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.272390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.272566 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.272882 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.272887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.272891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.273298 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:48.273614 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:48.273777 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 06:17:48.273993 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 06:17:48.275137 4958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:48.397543 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 4718
I20260811 06:17:48.424712 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 4970
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.2@KRBTEST.COM
I20260811 06:17:48.438279 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.425448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.22.121.2:32843)
0811 06:17:48.425635 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:48.425640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:48.425658 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:48.425768 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:48.425772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:48.425837 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:48.425915 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:48.427058 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.427809 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.429020 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.429403 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.429492 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.433474 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:48.433486 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:48.433501 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:48.433531 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:48.436113 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.436678 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.436684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.436686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.436807 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.437062 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.437065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.437067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.437395 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:48.437616 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:48.437805 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 06:17:48.437958 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:17:48.438920 5186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:48.559556 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 4970
I20260811 06:17:48.582659 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 5205
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[4494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.121.3@KRBTEST.COM
I20260811 06:17:48.596764 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:48.583395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33888 (local address 127.22.121.3:34641)
0811 06:17:48.583571 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:48.583575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:48.583613 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:17:48.583799 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:48.583802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:48.583864 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:48.583941 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:48.585249 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.585946 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.587158 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:48.587724 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:48.587818 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:48.591852 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:48.591868 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:48.591886 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:48.591919 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:48.594440 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.594996 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.595003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.595005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.595140 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:48.595451 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:48.595455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:48.595456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:48.595799 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:48.596030 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:48.596261 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 06:17:48.596444 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:17:48.597425 5441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:48.715029 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 5205
I20260811 06:17:48.739321 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 4520
I20260811 06:17:48.740401 4659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:48.852448 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 4520
2026-08-11T06:17:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6754 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[5671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[5671](info): set up 2 sockets
Aug 11 06:17:48 dist-test-slave-gjrz krb5kdc[5671](info): commencing operation
krb5kdc: starting...
W20260811 06:17:50.078282 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:50 dist-test-slave-gjrz krb5kdc[5671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:50.278354 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39009
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:50.584259 5695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:50.584527 5695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:50.584662 5695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:50.591481 5695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:50.591579 5695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:50.591656 5695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:50.591729 5695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:50.591794 5695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:50.601287 5695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:50.602396 5695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:50.603592 5695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:50.614420 5706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:50.614459 5709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:50.614562 5704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:50.615403 5695 server_base.cc:1034] running on GCE node
I20260811 06:17:50.616103 5695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:50.617319 5695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:50.618513 5695 hybrid_clock.cc:648] HybridClock initialized: now 1786429070618473 us; error 27 us; skew 500 ppm
I20260811 06:17:50.618795 5695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T06:17:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1832 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:17:50 dist-test-slave-gjrz krb5kdc[5719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:17:50 dist-test-slave-gjrz krb5kdc[5719](info): set up 2 sockets
Aug 11 06:17:50 dist-test-slave-gjrz krb5kdc[5719](info): commencing operation
krb5kdc: starting...
W20260811 06:17:51.921258 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:17:52 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:17:52.126240 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34539
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:52.432991 5737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:52.433255 5737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:52.433395 5737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:52.440231 5737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:52.440337 5737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:52.440412 5737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:52.440485 5737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:52.440546 5737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:52.449693 5737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:34539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:52.450819 5737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:52.452035 5737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:52.462843 5742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:52.462880 5744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:52.462800 5749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:52.463644 5737 server_base.cc:1034] running on GCE node
I20260811 06:17:52.464326 5737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:52.465515 5737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:52.466723 5737 hybrid_clock.cc:648] HybridClock initialized: now 1786429072466672 us; error 37 us; skew 500 ppm
I20260811 06:17:52.467020 5737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:52 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:52.473235 5737 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:52.474570 5737 webserver.cc:460] Webserver started at http://127.22.121.62:34375/ using document root <none> and password file <none>
I20260811 06:17:52.475184 5737 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:52.475318 5737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:52.475605 5737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:52.478256 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "345ee25c9e8246a8874efadb9a7c706b"
format_stamp: "Formatted at 2026-08-11 06:17:52 on dist-test-slave-gjrz"
server_key: "5fd6222f0e5b6118026665cc398fc5d1"
server_key_iv: "bf8c595d37b91a4c085c3402199fa592"
server_key_version: "encryptionkey@0"
I20260811 06:17:52.478955 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "345ee25c9e8246a8874efadb9a7c706b"
format_stamp: "Formatted at 2026-08-11 06:17:52 on dist-test-slave-gjrz"
server_key: "5fd6222f0e5b6118026665cc398fc5d1"
server_key_iv: "bf8c595d37b91a4c085c3402199fa592"
server_key_version: "encryptionkey@0"
I20260811 06:17:52.483758 5737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:17:52.486819 5762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:52.488006 5737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:17:52.488160 5737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "345ee25c9e8246a8874efadb9a7c706b"
format_stamp: "Formatted at 2026-08-11 06:17:52 on dist-test-slave-gjrz"
server_key: "5fd6222f0e5b6118026665cc398fc5d1"
server_key_iv: "bf8c595d37b91a4c085c3402199fa592"
server_key_version: "encryptionkey@0"
I20260811 06:17:52.488328 5737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:52.500327 5737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:52.503700 5737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:52.503931 5737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:52.520324 5737 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:34539
I20260811 06:17:52.520346 5867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:34539 every 8 connection(s)
I20260811 06:17:52.521498 5737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:52.522873 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 5737
I20260811 06:17:52.523135 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:17:52.523964 23012 external_mini_cluster.cc:1428] Setting key 75fc080524714b32284c4fe613a5effb
I20260811 06:17:52.525677 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:17:52.536252 5868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Bootstrap starting.
Aug 11 06:17:52 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:52.539207 5868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Neither blocks nor log segments found. Creating new log.
I20260811 06:17:52.540217 5868 log.cc:824] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:52.542689 5868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: No bootstrap required, opened a new log
I20260811 06:17:52.544195 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:52.525934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.22.121.62:34539)
0811 06:17:52.526688 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:52.526699 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:52.526736 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:17:52.528329 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:52.528349 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:52.528703 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:52.529047 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:52.530476 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:52.531864 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:52.533119 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:52.533783 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:52.533956 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:52.537729 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:52.537737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:52.537752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:52.537791 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:52.540568 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:52.541172 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:52.541178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:52.541184 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:52.541340 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:52.541703 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:52.541706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:52.541708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:52.542180 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:52.542427 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:52.542884 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 06:17:52.543323 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":336,"threads_started":1}
I20260811 06:17:52.549921 5868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:52.550247 5868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:17:52.550338 5868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345ee25c9e8246a8874efadb9a7c706b, State: Initialized, Role: FOLLOWER
I20260811 06:17:52.550822 5868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:52.551038 5868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:52.551170 5868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:52.551349 5868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:17:52.553411 5868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:52.553818 5868 leader_election.cc:302] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345ee25c9e8246a8874efadb9a7c706b; no voters:
I20260811 06:17:52.554333 5868 leader_election.cc:288] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:17:52.554426 5874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:17:52.555485 5874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 LEADER]: Becoming Leader. State: Replica: 345ee25c9e8246a8874efadb9a7c706b, State: Running, Role: LEADER
I20260811 06:17:52.555925 5874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:52.556233 5868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:52.558221 5876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345ee25c9e8246a8874efadb9a7c706b. Latest consensus state: current_term: 1 leader_uuid: "345ee25c9e8246a8874efadb9a7c706b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } } }
I20260811 06:17:52.558630 5876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:52.559082 5875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345ee25c9e8246a8874efadb9a7c706b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } } }
I20260811 06:17:52.559410 5875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:52.567864 5888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:52.572290 5888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:52.580163 5888 catalog_manager.cc:1347] Generated new cluster ID: 7b491cffee3547919f7c6d453e741fac
I20260811 06:17:52.580267 5888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:52.593302 5888 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:17:52.595067 5888 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:52.605006 5888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Generated new TSK 0
I20260811 06:17:52.605914 5888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:17:52.660837 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:52.973398 5914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:52.973654 5914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:52.973742 5914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:52.980516 5914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:52.980613 5914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:52.980726 5914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:52.989876 5914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:52.990926 5914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:52.992208 5914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:53.004253 5932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.004328 5931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.004292 5936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:53.005167 5914 server_base.cc:1034] running on GCE node
I20260811 06:17:53.005704 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:53.006520 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:53.007748 5914 hybrid_clock.cc:648] HybridClock initialized: now 1786429073007685 us; error 50 us; skew 500 ppm
I20260811 06:17:53.008052 5914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:53 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:53.010991 5914 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:17:53.012310 5914 webserver.cc:460] Webserver started at http://127.22.121.1:40805/ using document root <none> and password file <none>
I20260811 06:17:53.012943 5914 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:53.013057 5914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:53.013329 5914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:53.016062 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "65c28c3347064f668c7ee9fc9762b317"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "dc324d1324165ab18f2bcf458b51b6c1"
server_key_iv: "01fdae4b081ee38eab01ecc4eaa1f56c"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.016817 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "65c28c3347064f668c7ee9fc9762b317"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "dc324d1324165ab18f2bcf458b51b6c1"
server_key_iv: "01fdae4b081ee38eab01ecc4eaa1f56c"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.021523 5914 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 06:17:53.024721 5946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.025906 5914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:53.026059 5914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "65c28c3347064f668c7ee9fc9762b317"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "dc324d1324165ab18f2bcf458b51b6c1"
server_key_iv: "01fdae4b081ee38eab01ecc4eaa1f56c"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.026230 5914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:53.048321 5914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:53.051800 5914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:53.052052 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:53.052909 5914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:53.054108 5914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:53.054195 5914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.054307 5914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:53.054359 5914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.083431 5914 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:36499
I20260811 06:17:53.083490 6155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:36499 every 8 connection(s)
I20260811 06:17:53.084578 5914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:53.090497 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 5914
I20260811 06:17:53.090690 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:17:53.091315 23012 external_mini_cluster.cc:1428] Setting key f61867390e3c709ba501e56fa17b9ceb
Aug 11 06:17:53 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:53.104153 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:53.086866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:49273 (local address 127.22.121.62:34539)
0811 06:17:53.087367 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:53.087373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:53.088233 (+ 860us) server_negotiation.cc:407] Connection header received
0811 06:17:53.089927 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:53.089931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:53.089995 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:53.090092 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:53.091643 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.092602 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.094303 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.094875 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.095003 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:53.098281 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:53.098287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:53.098291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:53.098326 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:53.101302 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:53.102004 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:53.102010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:53.102013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:53.102165 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:53.102538 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:53.102543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:53.102547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:53.102808 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:53.102918 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:53.103689 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 06:17:53.103864 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":231,"threads_started":1}
I20260811 06:17:53.105293 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:53.087379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:49273)
0811 06:17:53.088078 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:53.088115 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:53.089699 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:53.090268 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:53.090275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:53.090808 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:53.091473 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:53.091494 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.092749 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.092755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:53.094157 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:53.094169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.095102 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.095108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:53.095350 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:53.096001 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:53.096023 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:53.098069 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:53.101437 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:53.101445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:53.101456 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:53.101864 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:53.102251 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:53.102254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:53.102257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:53.102426 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:53.102919 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:53.102927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:53.103335 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 06:17:53.103619 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:17:53.104102 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":254,"threads_started":1}
I20260811 06:17:53.106681 6156 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34539
I20260811 06:17:53.106946 6156 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:53.107686 6156 heartbeater.cc:511] Master 127.22.121.62:34539 requested a full tablet report, sending...
I20260811 06:17:53.109563 5800 ts_manager.cc:194] Registered new tserver with Master: 65c28c3347064f668c7ee9fc9762b317 (127.22.121.1:36499)
I20260811 06:17:53.111014 5800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:49273
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:17:53.142669 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:53.460757 6166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:53.461012 6166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:53.461148 6166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:53.468036 6166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:53.468138 6166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:53.468276 6166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:53.477350 6166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:53.478422 6166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:53.479655 6166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:53.491616 6179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.491671 6182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.492230 6178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:53.492766 6166 server_base.cc:1034] running on GCE node
I20260811 06:17:53.493439 6166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:53.494503 6166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:53.495764 6166 hybrid_clock.cc:648] HybridClock initialized: now 1786429073495683 us; error 44 us; skew 500 ppm
I20260811 06:17:53.496100 6166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:53 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:53.499974 6166 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:17:53.501294 6166 webserver.cc:460] Webserver started at http://127.22.121.2:33893/ using document root <none> and password file <none>
I20260811 06:17:53.501920 6166 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:53.502028 6166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:53.502313 6166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:53.504958 6166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "4ad43de243a14b55b05448648a4248f0"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "876463851ca280f4b912f12cf9afe19a"
server_key_iv: "d0d2924b7aa2649bcd8007fa9e3f359b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.505714 6166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "4ad43de243a14b55b05448648a4248f0"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "876463851ca280f4b912f12cf9afe19a"
server_key_iv: "d0d2924b7aa2649bcd8007fa9e3f359b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.510540 6166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:17:53.513691 6194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.514822 6166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:17:53.514966 6166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "4ad43de243a14b55b05448648a4248f0"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "876463851ca280f4b912f12cf9afe19a"
server_key_iv: "d0d2924b7aa2649bcd8007fa9e3f359b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.515141 6166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:53.536291 6166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:53.539780 6166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:53.540021 6166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:53.540874 6166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:53.542094 6166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:53.542181 6166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.542296 6166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:53.542351 6166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.572340 6166 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:36207
I20260811 06:17:53.572400 6417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:36207 every 8 connection(s)
I20260811 06:17:53.573503 6166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:53.579680 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 6166
I20260811 06:17:53.579876 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:17:53.580401 23012 external_mini_cluster.cc:1428] Setting key ad4e49af3688aade9338db06d385cbb0
Aug 11 06:17:53 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:53.593310 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:53.575819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49033 (local address 127.22.121.62:34539)
0811 06:17:53.576011 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:53.576017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:53.577192 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 06:17:53.578916 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:53.578920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:53.578983 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:53.579068 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:53.580945 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.581797 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.583842 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.584237 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.584330 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:53.588013 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:53.588018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:53.588023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:53.588055 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:53.590359 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:53.591123 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:53.591127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:53.591129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:53.591255 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:53.591719 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:53.591722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:53.591724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:53.591971 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:53.592118 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:53.592775 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 06:17:53.593000 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:17:53.594185 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:53.576353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:49033)
0811 06:17:53.577020 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:53.577066 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:53.578683 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:53.579286 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:53.579297 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:53.580005 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:53.580775 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:53.580800 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.581955 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.581960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:53.583708 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:53.583724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:53.584454 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:53.584460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:53.584726 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:53.585365 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:53.585398 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:53.587811 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:53.590490 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:53.590498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:53.590513 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:53.591010 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:53.591376 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:53.591381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:53.591384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:53.591626 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:53.592101 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:53.592107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:53.592388 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 06:17:53.592608 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 06:17:53.593064 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":246,"threads_started":1}
I20260811 06:17:53.595505 6418 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34539
I20260811 06:17:53.595781 6418 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:53.596415 6418 heartbeater.cc:511] Master 127.22.121.62:34539 requested a full tablet report, sending...
I20260811 06:17:53.597769 5800 ts_manager.cc:194] Registered new tserver with Master: 4ad43de243a14b55b05448648a4248f0 (127.22.121.2:36207)
I20260811 06:17:53.598531 5800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:49033
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:17:53.629990 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:53.937286 6423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:53.937533 6423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:53.937656 6423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:53.944501 6423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:53.944591 6423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:53.944739 6423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:53.953639 6423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:53.954686 6423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:53.955889 6423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:53.968318 6431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.968350 6430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:53.968293 6433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:53.969460 6423 server_base.cc:1034] running on GCE node
I20260811 06:17:53.970070 6423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:53.970916 6423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:53.972102 6423 hybrid_clock.cc:648] HybridClock initialized: now 1786429073972061 us; error 31 us; skew 500 ppm
I20260811 06:17:53.972375 6423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:53 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:53.975219 6423 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:17:53.976565 6423 webserver.cc:460] Webserver started at http://127.22.121.3:41501/ using document root <none> and password file <none>
I20260811 06:17:53.977174 6423 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:53.977268 6423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:53.977514 6423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:17:53.980242 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "e5928c1d914a42279ee781f78fa371ab"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "f7ceb76852642fdd5ef660fdad576c4b"
server_key_iv: "2a6ed27428ec7bedfe46800d21fbb38b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.980963 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "e5928c1d914a42279ee781f78fa371ab"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "f7ceb76852642fdd5ef660fdad576c4b"
server_key_iv: "2a6ed27428ec7bedfe46800d21fbb38b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.985834 6423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:17:53.988986 6447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:53.990145 6423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:17:53.990285 6423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "e5928c1d914a42279ee781f78fa371ab"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "f7ceb76852642fdd5ef660fdad576c4b"
server_key_iv: "2a6ed27428ec7bedfe46800d21fbb38b"
server_key_version: "encryptionkey@0"
I20260811 06:17:53.990453 6423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:54.010324 6423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:54.013782 6423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:54.014021 6423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:54.014879 6423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:54.016094 6423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:54.016168 6423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:54.016279 6423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:54.016330 6423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:54.045553 6423 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:40455
I20260811 06:17:54.045616 6653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:40455 every 8 connection(s)
I20260811 06:17:54.046687 6423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:54.048018 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 6423
I20260811 06:17:54.048184 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:17:54.048615 23012 external_mini_cluster.cc:1428] Setting key dde49d42784e05f774dc4ad7877d4661
Aug 11 06:17:54 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429073, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:17:54.065346 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:54.048999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:59951 (local address 127.22.121.62:34539)
0811 06:17:54.049208 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:54.049214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:54.050396 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 06:17:54.052524 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:54.052528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:54.052600 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:54.052670 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:54.053997 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.054718 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.056197 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.056678 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.056773 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:54.059937 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:54.059942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:54.059945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:54.059978 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:54.062562 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.063230 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.063234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.063237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.063388 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.063790 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.063793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.063795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.064049 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:54.064185 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:54.064803 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 06:17:54.065022 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:17:54.066154 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:54.049532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:59951)
0811 06:17:54.050253 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:54.050288 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:54.052292 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:54.052831 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:54.052837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:54.053304 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:54.053833 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:54.053849 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.054828 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.054831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:54.056076 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:54.056090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.056849 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.056854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:54.057049 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:54.057578 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:54.057616 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:54.059721 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:17:54.062696 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:54.062705 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:54.062718 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:54.063107 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:54.063489 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:17:54.063494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:17:54.063498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:17:54.063674 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:17:54.064173 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:17:54.064180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:17:54.064467 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 06:17:54.064684 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 06:17:54.065152 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":283,"threads_started":1}
I20260811 06:17:54.067330 6657 heartbeater.cc:347] Connected to a master server at 127.22.121.62:34539
I20260811 06:17:54.067608 6657 heartbeater.cc:464] Registering TS with master...
I20260811 06:17:54.068223 6657 heartbeater.cc:511] Master 127.22.121.62:34539 requested a full tablet report, sending...
I20260811 06:17:54.069492 5800 ts_manager.cc:194] Registered new tserver with Master: e5928c1d914a42279ee781f78fa371ab (127.22.121.3:40455)
I20260811 06:17:54.070246 5800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:59951
I20260811 06:17:54.075369 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.1@KRBTEST.COM" created.
I20260811 06:17:54.113750 6156 heartbeater.cc:503] Master 127.22.121.62:34539 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
Entry for principal oryx/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab.
WARNING: no policy specified for oryx/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
Entry for principal oryx/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab.
WARNING: no policy specified for oryx/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
Entry for principal oryx/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab.
I20260811 06:17:54.198009 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 5914
Aug 11 06:17:54 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:17:54.212924 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:54.199135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.22.121.1:36499)
0811 06:17:54.199786 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:54.199793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:54.199820 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:17:54.199882 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:54.199888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:54.200109 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:54.200257 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:54.201459 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.202493 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.203683 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.204243 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.204339 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:54.208009 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:54.208021 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:54.208035 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:54.208065 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:54.210697 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.211227 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.211233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.211236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.211386 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.211665 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.211667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.211670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.211978 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:54.212211 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:54.212394 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 06:17:54.212578 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":324,"threads_started":1}
I20260811 06:17:54.214145 6144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:54.294111 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 5914
I20260811 06:17:54.317183 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 6166
Aug 11 06:17:54 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:17:54.331965 6676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:54.317974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.22.121.2:36207)
0811 06:17:54.318478 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:54.318485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:54.318512 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:17:54.318589 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:54.318596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:54.318801 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:54.318931 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:54.320459 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.321456 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.322688 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.323213 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.323355 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:54.327095 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:54.327106 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:54.327120 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:54.327150 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:54.329755 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.330336 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.330342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.330344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.330450 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.330712 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.330715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.330717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.331024 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:54.331250 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:54.331486 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 06:17:54.331685 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260811 06:17:54.333164 6403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:54.412350 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 6166
I20260811 06:17:54.435936 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 6423
Aug 11 06:17:54 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:17:54.451118 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:54.436871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.22.121.3:40455)
0811 06:17:54.437482 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:54.437490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:54.437519 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:17:54.437581 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:54.437587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:54.437801 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:54.437927 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:54.439221 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.440308 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.441486 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:54.442040 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:54.442143 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:54.446235 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:54.446245 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:54.446261 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:54.446296 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:54.448913 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.449442 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.449448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.449450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.449578 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:54.449857 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:54.449860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:54.449862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:54.450181 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:54.450415 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:54.450586 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 06:17:54.450776 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":292,"threads_started":1}
I20260811 06:17:54.452349 6642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:54.546581 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 6423
I20260811 06:17:54.564953 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 5737
I20260811 06:17:54.566051 5859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:17:54.664785 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 5737
I20260811 06:17:54.693508 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34539
--webserver_interface=127.22.121.62
--webserver_port=34375
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:17:55.006619 6683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:55.006894 6683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:55.007027 6683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:55.014047 6683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:17:55.014149 6683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:55.014225 6683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:55.014295 6683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:17:55.014353 6683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:17:55.023890 6683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:34539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=34375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:55.025095 6683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:55.026363 6683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:55.037941 6699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.037916 6703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.037986 6700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:55.039079 6683 server_base.cc:1034] running on GCE node
I20260811 06:17:55.039839 6683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:55.041200 6683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:55.042388 6683 hybrid_clock.cc:648] HybridClock initialized: now 1786429075042332 us; error 44 us; skew 500 ppm
I20260811 06:17:55.042708 6683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:55 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429075, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:55.045603 6683 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:17:55.046991 6683 webserver.cc:460] Webserver started at http://127.22.121.62:34375/ using document root <none> and password file <none>
I20260811 06:17:55.047672 6683 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:55.047801 6683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:55.053194 6683 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 06:17:55.055830 6724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.057018 6683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:17:55.057214 6683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "345ee25c9e8246a8874efadb9a7c706b"
format_stamp: "Formatted at 2026-08-11 06:17:52 on dist-test-slave-gjrz"
server_key: "5fd6222f0e5b6118026665cc398fc5d1"
server_key_iv: "bf8c595d37b91a4c085c3402199fa592"
server_key_version: "encryptionkey@0"
I20260811 06:17:55.058123 6683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:55.085122 6683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:55.088614 6683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:55.088846 6683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:55.105466 6683 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:34539
I20260811 06:17:55.105484 6822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:34539 every 8 connection(s)
I20260811 06:17:55.106783 6683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:17:55.112856 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 6683
I20260811 06:17:55.113632 6823 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:17:55.113875 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:36499
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:34539
--webserver_port=40805
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:17:55.117970 6823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Bootstrap starting.
I20260811 06:17:55.137660 6823 log.cc:824] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:17:55.147111 6823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:17:55.147656 6823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Bootstrap complete.
I20260811 06:17:55.156467 6823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:55.157260 6823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345ee25c9e8246a8874efadb9a7c706b, State: Initialized, Role: FOLLOWER
I20260811 06:17:55.157820 6823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:55.158043 6823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:17:55.158146 6823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:17:55.158308 6823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:17:55.162166 6823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:55.162639 6823 leader_election.cc:302] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345ee25c9e8246a8874efadb9a7c706b; no voters:
I20260811 06:17:55.163226 6823 leader_election.cc:288] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:17:55.163380 6828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:17:55.165268 6823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:17:55.170002 6828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [term 2 LEADER]: Becoming Leader. State: Replica: 345ee25c9e8246a8874efadb9a7c706b, State: Running, Role: LEADER
I20260811 06:17:55.170579 6828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } }
I20260811 06:17:55.173126 6830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "345ee25c9e8246a8874efadb9a7c706b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } } }
I20260811 06:17:55.173508 6830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:55.174029 6831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345ee25c9e8246a8874efadb9a7c706b. Latest consensus state: current_term: 2 leader_uuid: "345ee25c9e8246a8874efadb9a7c706b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345ee25c9e8246a8874efadb9a7c706b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 34539 } } }
I20260811 06:17:55.174304 6831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b [sys.catalog]: This master's current role is: LEADER
I20260811 06:17:55.184635 6844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:17:55.192121 6844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:17:55.193612 6844 catalog_manager.cc:1259] Loaded cluster ID: 7b491cffee3547919f7c6d453e741fac
I20260811 06:17:55.193768 6844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:17:55.197717 6844 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:17:55.199147 6844 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 345ee25c9e8246a8874efadb9a7c706b: Loaded TSK: 0
I20260811 06:17:55.200809 6844 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:17:55.444190 6826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:55.444450 6826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:55.444586 6826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:55.451570 6826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:55.451675 6826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:55.451817 6826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:17:55.461108 6826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.1.keytab
--principal=oryx/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:36499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=40805
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:55.462396 6826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:55.463696 6826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:55.477739 6863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.478312 6861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.478641 6865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:55.479221 6826 server_base.cc:1034] running on GCE node
I20260811 06:17:55.479797 6826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:55.480690 6826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:55.481899 6826 hybrid_clock.cc:648] HybridClock initialized: now 1786429075481827 us; error 54 us; skew 500 ppm
I20260811 06:17:55.482182 6826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:55 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429075, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:55.485278 6826 init.cc:377] Logged in from keytab as oryx/127.22.121.1@KRBTEST.COM (short username oryx)
I20260811 06:17:55.486703 6826 webserver.cc:460] Webserver started at http://127.22.121.1:40805/ using document root <none> and password file <none>
I20260811 06:17:55.487387 6826 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:55.487519 6826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:55.493002 6826 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 06:17:55.496328 6878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.498443 6826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:17:55.498603 6826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "65c28c3347064f668c7ee9fc9762b317"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "dc324d1324165ab18f2bcf458b51b6c1"
server_key_iv: "01fdae4b081ee38eab01ecc4eaa1f56c"
server_key_version: "encryptionkey@0"
I20260811 06:17:55.499876 6826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:55.532913 6826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:55.537451 6826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:55.537707 6826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:55.538748 6826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:55.540369 6826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:55.540489 6826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.540601 6826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:55.540681 6826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.569826 6826 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:36499
I20260811 06:17:55.569916 7079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:36499 every 8 connection(s)
I20260811 06:17:55.571137 6826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:17:55.578827 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 6826
I20260811 06:17:55.579473 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:36207
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:34539
--webserver_port=33893
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:17:55 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:55 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:17:55.892033 7087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:55.892262 7087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:55.892376 7087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:55.899135 7087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:55.899228 7087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:55.899385 7087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:17:55.908553 7087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.2.keytab
--principal=oryx/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:36207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=33893
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:55.909626 7087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:55.910801 7087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:55.925529 7102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.925688 7104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:55.926281 7108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:55.927446 7087 server_base.cc:1034] running on GCE node
I20260811 06:17:55.927997 7087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:55.928875 7087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:55.930080 7087 hybrid_clock.cc:648] HybridClock initialized: now 1786429075930021 us; error 55 us; skew 500 ppm
I20260811 06:17:55.930365 7087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:55 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429075, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:55.933668 7087 init.cc:377] Logged in from keytab as oryx/127.22.121.2@KRBTEST.COM (short username oryx)
I20260811 06:17:55.935089 7087 webserver.cc:460] Webserver started at http://127.22.121.2:33893/ using document root <none> and password file <none>
I20260811 06:17:55.935738 7087 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:55.935866 7087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:55.941120 7087 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:17:55.943863 7119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.945214 7087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:55.945374 7087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "4ad43de243a14b55b05448648a4248f0"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "876463851ca280f4b912f12cf9afe19a"
server_key_iv: "d0d2924b7aa2649bcd8007fa9e3f359b"
server_key_version: "encryptionkey@0"
I20260811 06:17:55.946264 7087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:55.970467 7087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:55.974233 7087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:55.974467 7087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:55.975438 7087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:55.976738 7087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:55.976833 7087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:55.976933 7087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:55.977015 7087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:56.010380 7087 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:36207
I20260811 06:17:56.010457 7318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:36207 every 8 connection(s)
I20260811 06:17:56.011839 7087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:17:56.021857 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 7087
I20260811 06:17:56.022646 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:40455
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:34539
--webserver_port=41501
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:46179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:17:56.350704 7328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:17:56.350953 7328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:17:56.351116 7328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:17:56.358032 7328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:17:56.358141 7328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:17:56.358294 7328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:17:56.367959 7328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/krb5kdc/oryx_127.22.121.3.keytab
--principal=oryx/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:40455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=41501
--tserver_master_addrs=127.22.121.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:17:56.369422 7328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:17:56.370613 7328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:17:56.384703 7337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:56.385327 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:17:56.397056 7343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:17:56.397135 7328 server_base.cc:1034] running on GCE node
I20260811 06:17:56.397714 7328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:17:56.399508 7328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:17:56.400753 7328 hybrid_clock.cc:648] HybridClock initialized: now 1786429076400699 us; error 47 us; skew 500 ppm
I20260811 06:17:56.401043 7328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429076, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:17:56.404078 7328 init.cc:377] Logged in from keytab as oryx/127.22.121.3@KRBTEST.COM (short username oryx)
I20260811 06:17:56.405798 7328 webserver.cc:460] Webserver started at http://127.22.121.3:41501/ using document root <none> and password file <none>
I20260811 06:17:56.406515 7328 fs_manager.cc:359] Metadata directory not provided
I20260811 06:17:56.406644 7328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:17:56.411640 7328 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:17:56.414103 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:56.415400 7328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:17:56.415548 7328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "e5928c1d914a42279ee781f78fa371ab"
format_stamp: "Formatted at 2026-08-11 06:17:53 on dist-test-slave-gjrz"
server_key: "f7ceb76852642fdd5ef660fdad576c4b"
server_key_iv: "2a6ed27428ec7bedfe46800d21fbb38b"
server_key_version: "encryptionkey@0"
I20260811 06:17:56.416412 7328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:17:56.435084 7328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:17:56.438865 7328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:17:56.439108 7328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:17:56.440074 7328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:17:56.441392 7328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:17:56.441486 7328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:56.441592 7328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:17:56.441665 7328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:17:56.482033 7328 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:40455
I20260811 06:17:56.482095 7575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:40455 every 8 connection(s)
I20260811 06:17:56.483440 7328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:17:56.485569 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 7328
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:56.553727 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:56.486636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.22.121.62:34539)
0811 06:17:56.504999 (+ 18363us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:56.505006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:56.528421 (+ 23415us) server_negotiation.cc:407] Connection header received
0811 06:17:56.528610 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:56.528615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:56.528676 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:56.528771 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:56.530826 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.532011 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.534738 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.535359 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.535485 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.539808 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:17:56.539820 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:17:56.539839 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:17:56.539884 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:17:56.543922 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:56.544679 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:56.544687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:56.544695 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:56.544901 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:17:56.550056 (+ 5155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:17:56.550062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:17:56.550064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:17:56.550607 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:17:56.550996 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 06:17:56.552264 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0811 06:17:56.552709 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18228,"thread_start_us":18424,"threads_started":1}
W20260811 06:17:56.855123 7086 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.271s user 0.005s sys 0.008s
W20260811 06:17:56.856882 7085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:55.573878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:52473 (local address 127.22.121.62:34539)
0811 06:17:55.574527 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:55.574538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:55.574879 (+ 341us) server_negotiation.cc:407] Connection header received
0811 06:17:55.577116 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:55.577128 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:55.577434 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:55.577716 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:55.579017 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:55.580361 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:55.581847 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:55.582502 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:55.582669 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.856526 (+1273857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:52473: BlockingRecv error: recv got EOF from 127.22.121.1:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":236,"threads_started":1}
I20260811 06:17:56.857914 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:55.573978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:52473)
0811 06:17:55.574712 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:55.574759 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:55.576620 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:55.577857 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:55.577864 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:55.578322 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:55.578826 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:55.578843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:55.580524 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:55.580529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:55.581705 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:55.581718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:55.582717 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:55.582725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:55.582986 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:55.583589 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:55.583609 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:56.856202 (+1272593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":899,"threads_started":1}
W20260811 06:17:56.859068 7080 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:34539 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.121.62:34539: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:56.872421 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:56.859868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:51373)
0811 06:17:56.860152 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:56.860173 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:56.860309 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:56.860676 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:56.860681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:56.861224 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:56.861491 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.861504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.863180 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.863186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.864862 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.864878 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.866146 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.866154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.866379 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.867472 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:56.867500 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:56.872120 (+ 4620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260811 06:17:56.872637 7085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:56.859930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:51373 (local address 127.22.121.62:34539)
0811 06:17:56.860105 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:56.860112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:56.860249 (+ 137us) server_negotiation.cc:407] Connection header received
0811 06:17:56.860446 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:56.860452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:56.860539 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:56.860642 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:56.861627 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.863027 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.865048 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.865738 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.865864 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.872368 (+ 6504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:51373: BlockingRecv error: recv got EOF from 127.22.121.1:51373 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:56 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:56.885103 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:56.873448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:58709)
0811 06:17:56.873827 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:56.873848 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:56.873968 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:56.874340 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:56.874344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:56.874753 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:56.874960 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.874971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.876424 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.876429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.878673 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.878690 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.879725 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.879731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.879935 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.880586 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:56.880608 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:56.884834 (+ 4226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267}
W20260811 06:17:56.885320 7085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:56.873528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:58709 (local address 127.22.121.62:34539)
0811 06:17:56.873694 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:56.873700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:56.873935 (+ 235us) server_negotiation.cc:407] Connection header received
0811 06:17:56.874076 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:56.874082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:56.874187 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:56.874293 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:56.875105 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.876289 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.878819 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.879566 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.879696 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.885046 (+ 5350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:58709: BlockingRecv error: recv got EOF from 127.22.121.1:58709 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 06:17:56.885597 7080 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:17:57.215220 7327 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.190s user 0.004s sys 0.010s
W20260811 06:17:57.216686 7322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:56.014370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:42247 (local address 127.22.121.62:34539)
0811 06:17:56.014866 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:56.014872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:56.016000 (+ 1128us) server_negotiation.cc:407] Connection header received
0811 06:17:56.018849 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:56.018853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:56.018916 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:56.019012 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:56.021149 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.021923 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.023623 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.024037 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.024149 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.216458 (+1192309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:42247: BlockingRecv error: recv got EOF from 127.22.121.2:42247 (error 108)
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":228,"threads_started":1}
I20260811 06:17:57.217748 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:56.014857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:42247)
0811 06:17:56.015818 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:56.015873 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:56.018547 (+ 2674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:56.019321 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:56.019334 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:56.020124 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:56.020967 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.020992 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.022116 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.022120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.023473 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.023486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.024250 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.024259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.024495 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.025084 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:56.025106 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.216343 (+1191237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":328,"threads_started":1}
W20260811 06:17:57.218729 7320 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:34539 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.121.62:34539: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:57.230083 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:57.219354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:37493)
0811 06:17:57.219645 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:57.219664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:57.219779 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:57.220320 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:57.220326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:57.220894 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:57.221133 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.221147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.222168 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.222174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.223883 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.223901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.224575 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.224582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.224778 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.225411 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:57.225436 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.229776 (+ 4340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149}
W20260811 06:17:57.230105 7322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:57.219434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:37493 (local address 127.22.121.62:34539)
0811 06:17:57.219618 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:57.219624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:57.219730 (+ 106us) server_negotiation.cc:407] Connection header received
0811 06:17:57.219887 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:57.219893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:57.219984 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:57.220094 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:57.221270 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.222029 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.224029 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.224410 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.224502 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.229914 (+ 5412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:37493: BlockingRecv error: recv got EOF from 127.22.121.2:37493 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:57.240119 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:57.230912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:45019)
0811 06:17:57.231209 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:57.231227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:57.231351 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:57.231647 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:57.231652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:57.231946 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:57.232087 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.232095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.233327 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.233333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.235044 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.235063 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.235789 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.235796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.236003 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.236587 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:57.236605 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.239840 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199}
W20260811 06:17:57.240156 7322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:57.231055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:45019 (local address 127.22.121.62:34539)
0811 06:17:57.231192 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:57.231197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:57.231288 (+ 91us) server_negotiation.cc:407] Connection header received
0811 06:17:57.231468 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:57.231472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:57.231524 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:57.231599 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:57.232228 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.233178 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.235191 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.235656 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.235747 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.239993 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:45019: BlockingRecv error: recv got EOF from 127.22.121.2:45019 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 06:17:57.240414 7320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:17:57.596704 7590 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.055s user 0.005s sys 0.009s
W20260811 06:17:57.597771 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:56.494430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:42233 (local address 127.22.121.62:34539)
0811 06:17:56.522180 (+ 27750us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:56.522187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:56.527317 (+ 5130us) server_negotiation.cc:407] Connection header received
0811 06:17:56.527400 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:56.527406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:56.527498 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:56.527601 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:56.535532 (+ 7931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.536527 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.539342 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.539929 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.540025 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.597579 (+1057554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:42233: BlockingRecv error: recv got EOF from 127.22.121.3:42233 (error 108)
Metrics: {"server-negotiator.queue_time_us":27615,"thread_start_us":27775,"threads_started":1}
I20260811 06:17:57.598348 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:56.493929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:42233)
0811 06:17:56.523466 (+ 29537us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:56.523527 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:56.526172 (+ 2645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:56.531129 (+ 4957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:56.531140 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:56.531926 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:56.535316 (+ 3390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.535343 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.537149 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.537155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.539151 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:56.539166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:56.540358 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:56.540365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:56.540672 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:56.541261 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:56.541290 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.597417 (+1056127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29201,"thread_start_us":32552,"threads_started":1}
W20260811 06:17:57.599280 7576 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:34539 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.121.62:34539: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:57.611111 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:57.599870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:60109)
0811 06:17:57.600056 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:57.600076 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:57.600188 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:57.600477 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:57.600482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:57.600839 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:57.600999 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.601008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.602006 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.602010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.602958 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.602970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.603701 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.603707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.603843 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.607791 (+ 3948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:57.607809 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.610852 (+ 3043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:17:57.611259 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:57.599928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:60109 (local address 127.22.121.62:34539)
0811 06:17:57.600107 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:57.600113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:57.600131 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:17:57.600297 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:57.600302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:57.600360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:57.600436 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:57.601134 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.601868 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.603099 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.603532 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.603636 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.611050 (+ 7414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:60109: BlockingRecv error: recv got EOF from 127.22.121.3:60109 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:57.618777 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:57.611733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:37029)
0811 06:17:57.611869 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:57.611887 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:57.611983 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:57.612283 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:57.612286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:57.612547 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:57.612706 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.612714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.613612 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.613615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.614505 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.614515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.615315 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.615322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.615498 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.615923 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:57.615938 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.618613 (+ 2675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 06:17:57.619002 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:57.611841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37029 (local address 127.22.121.62:34539)
0811 06:17:57.612068 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:57.612079 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:57.612093 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:17:57.612128 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:57.612131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:57.612180 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:57.612252 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:57.612805 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.613494 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.614610 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.615166 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.615257 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.618778 (+ 3521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:37029: BlockingRecv error: recv got EOF from 127.22.121.3:37029 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
W20260811 06:17:57.619004 7576 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:57 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:57.894656 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:57.886102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:33217)
0811 06:17:57.886617 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:57.886633 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:57.886763 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:57.887083 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:57.887088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:57.887529 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:57.887680 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.887689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.888706 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.888710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.889889 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:57.889904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.890707 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.890712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:57.890832 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.891356 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:57.891373 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:57.894419 (+ 3046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":288,"threads_started":1}
W20260811 06:17:57.894784 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:57.886248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:33217 (local address 127.22.121.62:34539)
0811 06:17:57.886455 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:57.886460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:57.886712 (+ 252us) server_negotiation.cc:407] Connection header received
0811 06:17:57.886870 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:57.886876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:57.886945 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:57.887023 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:57.887829 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.888536 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.890040 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:57.890553 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:57.890650 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:57.894610 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:33217: BlockingRecv error: recv got EOF from 127.22.121.1:33217 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:58.249697 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:58.240917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:48037)
0811 06:17:58.241445 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:58.241461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:58.241592 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:58.241920 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:58.241924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:58.242344 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:58.242513 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.242523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.243595 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.243600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.244549 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.244562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.245253 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.245259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.245378 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.245894 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:58.245912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:58.249426 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":291,"threads_started":1}
W20260811 06:17:58.249832 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:58.240983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:48037 (local address 127.22.121.62:34539)
0811 06:17:58.241136 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:58.241141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:58.241528 (+ 387us) server_negotiation.cc:407] Connection header received
0811 06:17:58.241673 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:58.241676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:58.241736 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:58.241812 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:58.242653 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.243431 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.244687 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.245106 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.245201 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.249626 (+ 4425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:48037: BlockingRecv error: recv got EOF from 127.22.121.2:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:58.627843 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:58.619546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:48149)
0811 06:17:58.620064 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:58.620082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:58.620197 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:58.620456 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:58.620459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:58.620899 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:58.621072 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.621084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.622013 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.622016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.622950 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.622961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.623661 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.623665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.623785 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.624328 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:58.624345 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:58.627515 (+ 3170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":281,"threads_started":1}
W20260811 06:17:58.627916 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:58.619662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:48149 (local address 127.22.121.62:34539)
0811 06:17:58.619873 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:58.619879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:58.620156 (+ 277us) server_negotiation.cc:407] Connection header received
0811 06:17:58.620286 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:58.620289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:58.620350 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:58.620425 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:58.621190 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.621905 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.623056 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.623519 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.623611 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.627723 (+ 4112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:48149: BlockingRecv error: recv got EOF from 127.22.121.3:48149 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:58 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:58.905208 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:58.895526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:45377)
0811 06:17:58.895980 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:58.895996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:58.896120 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:58.896376 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:58.896379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:58.896797 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:58.896978 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.896986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.897903 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.897906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.898823 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:58.898833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.899694 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.899700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:58.899878 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.900512 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:58.900537 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:58.904873 (+ 4336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":258,"threads_started":1}
W20260811 06:17:58.905462 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:58.895591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:45377 (local address 127.22.121.62:34539)
0811 06:17:58.895800 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:58.895807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:58.896067 (+ 260us) server_negotiation.cc:407] Connection header received
0811 06:17:58.896208 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:58.896212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:58.896273 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:58.896347 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:58.897087 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.897788 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.898945 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:58.899534 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:58.899640 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:58.905173 (+ 5533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:45377: BlockingRecv error: recv got EOF from 127.22.121.1:45377 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:59.259110 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:59.250506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:55831)
0811 06:17:59.250994 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:59.251008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:59.251156 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:59.251501 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:59.251505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:59.251931 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:59.252126 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.252135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.253144 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.253150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.254062 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.254085 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.254775 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.254782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.254947 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.255474 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:59.255490 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:59.258876 (+ 3386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":282,"threads_started":1}
W20260811 06:17:59.259227 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:59.250615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55831 (local address 127.22.121.62:34539)
0811 06:17:59.250833 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:59.250839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:59.251091 (+ 252us) server_negotiation.cc:407] Connection header received
0811 06:17:59.251287 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:59.251291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:59.251354 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:59.251433 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:59.252270 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.253001 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.254212 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.254627 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.254734 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.259047 (+ 4313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:55831: BlockingRecv error: recv got EOF from 127.22.121.2:55831 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:59.637025 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:59.628693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:53789)
0811 06:17:59.629162 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:59.629179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:59.629292 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:59.629546 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:59.629549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:59.629937 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:59.630128 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.630137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.631087 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.631090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.632054 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.632067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.632648 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.632652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.632797 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.633308 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:59.633325 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:59.636782 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":255,"threads_started":1}
W20260811 06:17:59.637130 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:59.628799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:53789 (local address 127.22.121.62:34539)
0811 06:17:59.628947 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:59.628951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:59.629241 (+ 290us) server_negotiation.cc:407] Connection header received
0811 06:17:59.629377 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:59.629380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:59.629441 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:59.629517 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:59.630235 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.630968 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.632162 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.632538 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.632633 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.636944 (+ 4311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:53789: BlockingRecv error: recv got EOF from 127.22.121.3:53789 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:17:59 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:17:59.914861 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:17:59.906219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:33049)
0811 06:17:59.906683 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 06:17:59.906699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:17:59.906810 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:17:59.907070 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:17:59.907074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:17:59.907466 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:17:59.907654 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.907664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.908651 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.908656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.909584 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:17:59.909596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.910227 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.910233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:17:59.910402 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.911013 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:17:59.911036 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:17:59.914529 (+ 3493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":239,"threads_started":1}
W20260811 06:17:59.915004 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:17:59.906299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:33049 (local address 127.22.121.62:34539)
0811 06:17:59.906506 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:17:59.906512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:17:59.906773 (+ 261us) server_negotiation.cc:407] Connection header received
0811 06:17:59.906898 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:17:59.906902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:17:59.906963 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:17:59.907039 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:17:59.907770 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.908502 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.909692 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:17:59.910094 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:17:59.910180 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:17:59.914780 (+ 4600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:33049: BlockingRecv error: recv got EOF from 127.22.121.1:33049 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:00.268630 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:00.260027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:57017)
0811 06:18:00.260509 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:00.260523 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:00.260659 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:00.260984 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:00.260989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:00.261387 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:00.261550 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.261560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.262675 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.262680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.263614 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.263628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.264326 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.264332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.264502 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.265041 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:00.265059 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:00.268303 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
W20260811 06:18:00.268750 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:00.260044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:57017 (local address 127.22.121.62:34539)
0811 06:18:00.260218 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:00.260224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:00.260602 (+ 378us) server_negotiation.cc:407] Connection header received
0811 06:18:00.260771 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:00.260777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:00.260848 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:00.260947 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:00.261706 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.262523 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.263751 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.264188 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.264299 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.268522 (+ 4223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:57017: BlockingRecv error: recv got EOF from 127.22.121.2:57017 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:00.647378 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:00.637840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:48885)
0811 06:18:00.639531 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:00.639547 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:00.639683 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:00.639996 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:00.640001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:00.640393 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:00.640552 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.640561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.641542 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.641548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.642466 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.642478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.643159 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.643165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.643373 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.643910 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:00.643928 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:00.647052 (+ 3124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1589,"thread_start_us":1462,"threads_started":1}
W20260811 06:18:00.647495 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:00.637921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:48885 (local address 127.22.121.62:34539)
0811 06:18:00.638059 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:00.638063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:00.639640 (+ 1577us) server_negotiation.cc:407] Connection header received
0811 06:18:00.639799 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:00.639803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:00.639864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:00.639940 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:00.640707 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.641405 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.642609 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.643023 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.643138 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.647287 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:48885: BlockingRecv error: recv got EOF from 127.22.121.3:48885 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:00 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:00.924212 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:00.915761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:35499)
0811 06:18:00.916246 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:00.916263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:00.916377 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:00.916661 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:00.916664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:00.917037 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:00.917203 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.917211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.918129 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.918132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.919181 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:00.919195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.919807 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.919811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:00.919932 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.920526 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:00.920544 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:00.923956 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
W20260811 06:18:00.924347 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:00.915829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:35499 (local address 127.22.121.62:34539)
0811 06:18:00.915992 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:00.915996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:00.916327 (+ 331us) server_negotiation.cc:407] Connection header received
0811 06:18:00.916463 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:00.916467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:00.916533 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:00.916608 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:00.917306 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.918022 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.919310 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:00.919701 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:00.919793 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:00.924144 (+ 4351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:35499: BlockingRecv error: recv got EOF from 127.22.121.1:35499 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:01.277901 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:01.269526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:37791)
0811 06:18:01.270040 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:01.270056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:01.270166 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:01.270448 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:01.270451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:01.270811 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:01.271003 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.271014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.272033 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.272037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.273073 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.273086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.273759 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.273764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.273922 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.274485 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:01.274501 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:01.277658 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":268,"threads_started":1}
W20260811 06:18:01.278046 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:01.269563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:37791 (local address 127.22.121.62:34539)
0811 06:18:01.269728 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:01.269732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:01.270118 (+ 386us) server_negotiation.cc:407] Connection header received
0811 06:18:01.270266 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:01.270269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:01.270329 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:01.270399 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:01.271125 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.271923 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.273196 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.273626 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.273718 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.277837 (+ 4119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:37791: BlockingRecv error: recv got EOF from 127.22.121.2:37791 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:01.656925 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:01.648401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:47379)
0811 06:18:01.648890 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:01.648909 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:01.649057 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:01.649470 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:01.649473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:01.649834 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:01.650000 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.650009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.651102 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.651106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.652111 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.652123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.652758 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.652762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.652888 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.653412 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:01.653429 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:01.656677 (+ 3248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":322,"threads_started":1}
W20260811 06:18:01.657088 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:01.648512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:47379 (local address 127.22.121.62:34539)
0811 06:18:01.648719 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:01.648724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:01.649008 (+ 284us) server_negotiation.cc:407] Connection header received
0811 06:18:01.649237 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:01.649244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:01.649328 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:01.649420 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:01.650192 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.650977 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.652247 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.652651 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.652746 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.656868 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:47379: BlockingRecv error: recv got EOF from 127.22.121.3:47379 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:01 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:01.933431 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:01.925059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:42625)
0811 06:18:01.925568 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:01.925592 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:01.925705 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:01.925972 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:01.925975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:01.926370 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:01.926524 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.926532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.927579 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.927583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.928485 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:01.928496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.929167 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.929171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:01.929287 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.929788 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:01.929804 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:01.933116 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
W20260811 06:18:01.933542 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:01.925143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42625 (local address 127.22.121.62:34539)
0811 06:18:01.925341 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:01.925347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:01.925678 (+ 331us) server_negotiation.cc:407] Connection header received
0811 06:18:01.925791 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:01.925794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:01.925864 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:01.925949 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:01.926671 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.927472 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.928617 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:01.929061 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:01.929154 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:01.933319 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:42625: BlockingRecv error: recv got EOF from 127.22.121.1:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:02.287604 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:02.279027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:37079)
0811 06:18:02.279533 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:02.279549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:02.279674 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:02.279986 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:02.279991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:02.280429 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:02.280584 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.280596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.281599 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.281605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.282570 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.282584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.283352 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.283358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.283528 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.284087 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:02.284105 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:02.287294 (+ 3189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":276,"threads_started":1}
W20260811 06:18:02.287755 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:02.279225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:37079 (local address 127.22.121.62:34539)
0811 06:18:02.279415 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:02.279422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:02.279621 (+ 199us) server_negotiation.cc:407] Connection header received
0811 06:18:02.279788 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:02.279792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:02.279853 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:02.279935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:02.280727 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.281468 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.282711 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.283185 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.283314 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.287504 (+ 4190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:37079: BlockingRecv error: recv got EOF from 127.22.121.2:37079 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:02.670728 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:02.657748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:39927)
0811 06:18:02.658232 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:02.658247 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:02.658379 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:02.658700 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:02.658704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:02.659178 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:02.659366 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.659376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.660351 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.660357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.661340 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.661353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.662245 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.662250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.662415 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.667357 (+ 4942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:02.667374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:02.670420 (+ 3046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":249,"threads_started":1}
W20260811 06:18:02.670849 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:02.657827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:39927 (local address 127.22.121.62:34539)
0811 06:18:02.658005 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:02.658011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:02.658325 (+ 314us) server_negotiation.cc:407] Connection header received
0811 06:18:02.658495 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:02.658499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:02.658561 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:02.658649 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:02.659497 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.660213 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.661489 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.662097 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.662206 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.670631 (+ 8425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:39927: BlockingRecv error: recv got EOF from 127.22.121.3:39927 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:02 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:02.943035 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:02.934402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:41143)
0811 06:18:02.934827 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:02.934843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:02.934955 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:02.935215 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:02.935218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:02.935638 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:02.935803 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.935812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.937041 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.937045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.938153 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:02.938167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.938837 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.938842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:02.938965 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.939541 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:02.939559 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:02.942717 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":227,"threads_started":1}
W20260811 06:18:02.943192 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:02.934447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:41143 (local address 127.22.121.62:34539)
0811 06:18:02.934693 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:02.934700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:02.934922 (+ 222us) server_negotiation.cc:407] Connection header received
0811 06:18:02.935043 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:02.935046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:02.935110 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:02.935187 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:02.935918 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.936903 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.938290 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:02.938701 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:02.938797 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:02.942954 (+ 4157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:41143: BlockingRecv error: recv got EOF from 127.22.121.1:41143 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:03.296810 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:03.288509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:44025)
0811 06:18:03.288915 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:03.288931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:03.289044 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:03.289372 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:03.289376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:03.289782 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:03.289960 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.289970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.290903 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.290907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.291853 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.291865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.292538 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.292542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.292660 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.293175 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:03.293192 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:03.296579 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":211,"threads_started":1}
W20260811 06:18:03.296953 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:03.288546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:44025 (local address 127.22.121.62:34539)
0811 06:18:03.288737 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:03.288744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:03.289004 (+ 260us) server_negotiation.cc:407] Connection header received
0811 06:18:03.289164 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:03.289168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:03.289247 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:03.289326 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:03.290084 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.290795 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.291981 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.292421 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.292521 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.296779 (+ 4258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:44025: BlockingRecv error: recv got EOF from 127.22.121.2:44025 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:03.680184 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:03.671764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:56983)
0811 06:18:03.672173 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:03.672189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:03.672340 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:03.672649 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:03.672653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:03.673136 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:03.673342 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.673355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.674577 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.674582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.675527 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.675539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.676162 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.676166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.676284 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.676848 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:03.676867 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:03.679953 (+ 3086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":202,"threads_started":1}
W20260811 06:18:03.680338 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:03.671816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:56983 (local address 127.22.121.62:34539)
0811 06:18:03.672016 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:03.672023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:03.672297 (+ 274us) server_negotiation.cc:407] Connection header received
0811 06:18:03.672449 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:03.672454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:03.672516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:03.672604 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:03.673489 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.674454 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.675638 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.676040 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.676139 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.680140 (+ 4001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:56983: BlockingRecv error: recv got EOF from 127.22.121.3:56983 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:03 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:03.952397 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:03.943916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:53627)
0811 06:18:03.944520 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:03.944536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:03.944651 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:03.944905 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:03.944908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:03.945307 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:03.945486 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.945495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.946417 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.946421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.947400 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:03.947413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.948031 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.948035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:03.948155 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.948636 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:03.948652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:03.952129 (+ 3477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":282,"threads_started":1}
W20260811 06:18:03.952489 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:03.944000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:53627 (local address 127.22.121.62:34539)
0811 06:18:03.944188 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:03.944194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:03.944604 (+ 410us) server_negotiation.cc:407] Connection header received
0811 06:18:03.944737 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:03.944740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:03.944802 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:03.944878 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:03.945611 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.946307 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.947503 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:03.947919 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:03.948009 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:03.952311 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:53627: BlockingRecv error: recv got EOF from 127.22.121.1:53627 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:04.306280 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:04.297627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:48285)
0811 06:18:04.298274 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:04.298290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:04.298417 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:04.298672 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:04.298675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:04.299090 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:04.299308 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.299321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.300305 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.300310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.301269 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.301282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.301919 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.301923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.302039 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.302549 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:04.302565 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:04.305940 (+ 3375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":323,"threads_started":1}
W20260811 06:18:04.306442 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:04.297700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:48285 (local address 127.22.121.62:34539)
0811 06:18:04.297869 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:04.297874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:04.298352 (+ 478us) server_negotiation.cc:407] Connection header received
0811 06:18:04.298503 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:04.298506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:04.298566 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:04.298642 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:04.299429 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.300152 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.301380 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.301798 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.301899 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.306174 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:48285: BlockingRecv error: recv got EOF from 127.22.121.2:48285 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:04.689836 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:04.681001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:35967)
0811 06:18:04.681590 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:04.681606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:04.681735 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:04.682070 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:04.682087 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:04.682497 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:04.682673 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.682682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.683756 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.683760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.684712 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.684729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.685581 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.685587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.685757 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.686329 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:04.686347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:04.689494 (+ 3147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":254,"threads_started":1}
W20260811 06:18:04.689896 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:04.681109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:35967 (local address 127.22.121.62:34539)
0811 06:18:04.681297 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:04.681304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:04.681693 (+ 389us) server_negotiation.cc:407] Connection header received
0811 06:18:04.681845 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:04.681850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:04.681921 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:04.682001 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:04.682818 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.683629 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.684870 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.685405 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.685523 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.689703 (+ 4180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:35967: BlockingRecv error: recv got EOF from 127.22.121.3:35967 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:04 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:04.961493 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:04.953233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:43973)
0811 06:18:04.953711 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:04.953728 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:04.953838 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:04.954104 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:04.954107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:04.954497 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:04.954662 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.954671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.955650 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.955654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.956583 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:04.956593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.957209 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.957214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:04.957327 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.957894 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:04.957912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:04.961259 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":232,"threads_started":1}
W20260811 06:18:04.961668 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:04.953406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43973 (local address 127.22.121.62:34539)
0811 06:18:04.953629 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:04.953635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:04.953789 (+ 154us) server_negotiation.cc:407] Connection header received
0811 06:18:04.953925 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:04.953929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:04.953989 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:04.954063 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:04.954767 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.955546 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.956682 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:04.957062 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:04.957158 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:04.961420 (+ 4262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:43973: BlockingRecv error: recv got EOF from 127.22.121.1:43973 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:05.316015 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:05.307154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:51279)
0811 06:18:05.307668 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:05.307684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:05.307814 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:05.308115 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:05.308119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:05.308537 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:05.308721 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.308731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.309862 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.309866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.310799 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.310811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.311611 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.311618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.311757 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.312283 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:05.312299 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:05.315680 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
W20260811 06:18:05.316179 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:05.307245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:51279 (local address 127.22.121.62:34539)
0811 06:18:05.307449 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:05.307456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:05.307766 (+ 310us) server_negotiation.cc:407] Connection header received
0811 06:18:05.307925 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:05.307929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:05.307990 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:05.308070 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:05.308863 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.309715 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.310938 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.311421 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.311533 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.315924 (+ 4391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:51279: BlockingRecv error: recv got EOF from 127.22.121.2:51279 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:05.699188 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:05.690765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:50883)
0811 06:18:05.691227 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:05.691243 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:05.691385 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:05.691642 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:05.691645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:05.692042 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:05.692217 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.692226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.693154 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.693157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.694238 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.694251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.695149 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.695154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.695308 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.695818 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:05.695835 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:05.698935 (+ 3100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
W20260811 06:18:05.699391 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:05.690819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:50883 (local address 127.22.121.62:34539)
0811 06:18:05.691009 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:05.691016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:05.691336 (+ 320us) server_negotiation.cc:407] Connection header received
0811 06:18:05.691471 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:05.691474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:05.691536 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:05.691624 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:05.692320 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.693050 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.694374 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.694984 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.695094 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.699142 (+ 4048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:50883: BlockingRecv error: recv got EOF from 127.22.121.3:50883 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:05 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:05.970999 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:05.962446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:45633)
0811 06:18:05.962915 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:05.962931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:05.963040 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:05.963397 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:05.963401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:05.963783 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:05.963931 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.963939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.964860 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.964863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.965917 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:05.965931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.966564 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.966569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:05.966692 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.967217 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:05.967234 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:05.970694 (+ 3460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":237,"threads_started":1}
W20260811 06:18:05.971196 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:05.962538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:45633 (local address 127.22.121.62:34539)
0811 06:18:05.962696 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:05.962701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:05.962992 (+ 291us) server_negotiation.cc:407] Connection header received
0811 06:18:05.963171 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:05.963176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:05.963240 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:05.963343 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:05.964035 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.964756 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.966023 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:05.966440 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:05.966527 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:05.970975 (+ 4448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:45633: BlockingRecv error: recv got EOF from 127.22.121.1:45633 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:06.325615 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:06.317032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:50363)
0811 06:18:06.317526 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:06.317541 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:06.317681 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:06.318010 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:06.318015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:06.318430 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:06.318597 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.318607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.319758 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.319763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.320829 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.320842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.321554 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.321558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.321697 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.322219 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:06.322235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:06.325389 (+ 3154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":247,"threads_started":1}
W20260811 06:18:06.325794 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:06.317111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:50363 (local address 127.22.121.62:34539)
0811 06:18:06.317300 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:06.317307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:06.317619 (+ 312us) server_negotiation.cc:407] Connection header received
0811 06:18:06.317791 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:06.317796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:06.317865 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:06.317946 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:06.318744 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.319614 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.320991 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.321426 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.321518 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.325556 (+ 4038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:50363: BlockingRecv error: recv got EOF from 127.22.121.2:50363 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:06.708813 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:06.700076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:52705)
0811 06:18:06.700561 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:06.700577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:06.700687 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:06.700976 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:06.700979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:06.701357 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:06.701507 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.701516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.702471 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.702475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.703460 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.703473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.704235 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.704239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.704365 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.704950 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:06.704968 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:06.708586 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":238,"threads_started":1}
W20260811 06:18:06.708985 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:06.700125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:52705 (local address 127.22.121.62:34539)
0811 06:18:06.700289 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:06.700294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:06.700640 (+ 346us) server_negotiation.cc:407] Connection header received
0811 06:18:06.700771 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:06.700775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:06.700834 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:06.700909 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:06.701613 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.702332 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.703606 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.704095 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.704194 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.708779 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:52705: BlockingRecv error: recv got EOF from 127.22.121.3:52705 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:06 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:06.980545 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:06.971843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:58019)
0811 06:18:06.972445 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:06.972461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:06.972602 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:06.972994 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:06.972999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:06.973478 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:06.973641 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.973651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.974693 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.974699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.975736 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:06.975749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.976443 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.976449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:06.976612 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.977156 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:06.977173 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:06.980245 (+ 3072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":344,"threads_started":1}
W20260811 06:18:06.980698 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:06.971905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:58019 (local address 127.22.121.62:34539)
0811 06:18:06.972115 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:06.972122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:06.972528 (+ 406us) server_negotiation.cc:407] Connection header received
0811 06:18:06.972729 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:06.972735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:06.972827 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:06.972909 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:06.973787 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.974556 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.975870 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:06.976301 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:06.976410 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:06.980441 (+ 4031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:58019: BlockingRecv error: recv got EOF from 127.22.121.1:58019 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:07.335108 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:07.326472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:54157)
0811 06:18:07.326920 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:07.326936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:07.327049 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:07.327356 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:07.327359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:07.327737 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:07.327900 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.327909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.328822 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.328826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.329808 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.329821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.330905 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.330912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.331096 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.331775 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:07.331798 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:07.334857 (+ 3059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
W20260811 06:18:07.335222 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:07.326551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:54157 (local address 127.22.121.62:34539)
0811 06:18:07.326753 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:07.326759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:07.327042 (+ 283us) server_negotiation.cc:407] Connection header received
0811 06:18:07.327150 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:07.327154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:07.327218 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:07.327333 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:07.328011 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.328697 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.329980 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.330588 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.330871 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.335040 (+ 4169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:54157: BlockingRecv error: recv got EOF from 127.22.121.2:54157 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:07.721181 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:07.709748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:47413)
0811 06:18:07.710273 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:07.710289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:07.710407 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:07.710663 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:07.710666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:07.711029 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:07.711201 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.711210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.712225 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.712229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.713208 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.713221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.713845 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.713849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.713970 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.717413 (+ 3443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:07.717431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:07.720871 (+ 3440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":287,"threads_started":1}
W20260811 06:18:07.721330 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:07.709860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:47413 (local address 127.22.121.62:34539)
0811 06:18:07.710060 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:07.710065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:07.710351 (+ 286us) server_negotiation.cc:407] Connection header received
0811 06:18:07.710494 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:07.710497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:07.710558 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:07.710633 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:07.711354 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.712092 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.713316 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.713735 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.713820 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.721072 (+ 7252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:47413: BlockingRecv error: recv got EOF from 127.22.121.3:47413 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:07 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:07.990070 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:07.981538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:37537)
0811 06:18:07.982027 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:07.982042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:07.982187 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:07.982520 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:07.982525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:07.982962 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:07.983152 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.983161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.984188 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.984192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.985112 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:07.985124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.985832 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.985838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:07.986013 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.986591 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:07.986609 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:07.989853 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":256,"threads_started":1}
W20260811 06:18:07.990229 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:07.981621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:37537 (local address 127.22.121.62:34539)
0811 06:18:07.981806 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:07.981811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:07.982140 (+ 329us) server_negotiation.cc:407] Connection header received
0811 06:18:07.982301 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:07.982305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:07.982381 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:07.982466 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:07.983324 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.984056 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.985252 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:07.985685 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:07.985795 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:07.990027 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:37537: BlockingRecv error: recv got EOF from 127.22.121.1:37537 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:08.344691 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:08.335995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:58977)
0811 06:18:08.336453 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:08.336468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:08.336608 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:08.336918 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:08.336923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:08.337359 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:08.337537 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.337547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.338581 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.338586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.339565 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.339585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.340273 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.340279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.340449 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.341013 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:08.341031 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:08.344472 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
W20260811 06:18:08.344866 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:08.336085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:58977 (local address 127.22.121.62:34539)
0811 06:18:08.336276 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:08.336282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:08.336548 (+ 266us) server_negotiation.cc:407] Connection header received
0811 06:18:08.336722 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:08.336726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:08.336788 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:08.336870 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:08.337690 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.338445 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.339712 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.340126 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.340250 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.344650 (+ 4400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:58977: BlockingRecv error: recv got EOF from 127.22.121.2:58977 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:08.730188 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:08.722064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:55041)
0811 06:18:08.722547 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:08.722563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:08.722679 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:08.722989 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:08.722992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:08.723472 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:08.723664 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.723673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.724618 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.724622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.725529 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.725540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.726134 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.726137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.726256 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.726815 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:08.726831 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:08.729975 (+ 3144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":253,"threads_started":1}
W20260811 06:18:08.730396 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:08.722119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:55041 (local address 127.22.121.62:34539)
0811 06:18:08.722298 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:08.722304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:08.722623 (+ 319us) server_negotiation.cc:407] Connection header received
0811 06:18:08.722803 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:08.722808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:08.722878 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:08.722952 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:08.723774 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.724514 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.725644 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.726029 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.726113 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.730181 (+ 4068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:55041: BlockingRecv error: recv got EOF from 127.22.121.3:55041 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:08 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:08.999601 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:08.990913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:59453)
0811 06:18:08.991487 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:08.991503 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:08.991630 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:08.991908 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:08.991911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:08.992291 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:08.992462 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.992470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.993429 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.993433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.994342 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:08.994354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.995051 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.995055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:08.995181 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.995741 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:08.995758 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:08.999237 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":288,"threads_started":1}
W20260811 06:18:08.999703 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:08.990964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:59453 (local address 127.22.121.62:34539)
0811 06:18:08.991133 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:08.991138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:08.991569 (+ 431us) server_negotiation.cc:407] Connection header received
0811 06:18:08.991711 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:08.991715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:08.991786 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:08.991864 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:08.992569 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.993314 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.994470 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:08.994915 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:08.995004 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:08.999466 (+ 4462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:59453: BlockingRecv error: recv got EOF from 127.22.121.1:59453 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 06:18:09 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:09 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:09.353930 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:09.345499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:59183)
0811 06:18:09.345994 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:09.346010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:09.346145 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:09.346406 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:09.346409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:09.346802 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:09.346975 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:09.346984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.347955 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.347959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:09.349080 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:09.349095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.349756 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.349761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:09.349887 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:09.350435 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:09.350453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:09.353658 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":243,"threads_started":1}
W20260811 06:18:09.354128 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:09.345534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:59183 (local address 127.22.121.62:34539)
0811 06:18:09.345735 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:09.345742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:09.346084 (+ 342us) server_negotiation.cc:407] Connection header received
0811 06:18:09.346237 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:09.346240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:09.346301 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:09.346376 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:09.347082 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.347850 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.349209 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.349626 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.349716 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:09.353863 (+ 4147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:59183: BlockingRecv error: recv got EOF from 127.22.121.2:59183 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 06:18:09 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:09 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:09.739648 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:09.730996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:58311)
0811 06:18:09.731508 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:09.731530 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:09.731679 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:09.732019 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:09.732025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:09.732557 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:09.732788 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:09.732803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.733831 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.733835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:09.734757 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:09.734768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.735423 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.735427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:09.735555 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:09.736198 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:09.736214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:09.739310 (+ 3096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":268,"threads_started":1}
W20260811 06:18:09.739749 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:09.731061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58311 (local address 127.22.121.62:34539)
0811 06:18:09.731298 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:09.731304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:09.731618 (+ 314us) server_negotiation.cc:407] Connection header received
0811 06:18:09.731793 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:09.731798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:09.731879 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:09.731969 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:09.732944 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.733717 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.734861 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:09.735282 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:09.735370 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:09.739525 (+ 4155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:58311: BlockingRecv error: recv got EOF from 127.22.121.3:58311 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:10.009001 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:10.000677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:40757)
0811 06:18:10.001099 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:10.001115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:10.001228 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:10.001511 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:10.001514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:10.001922 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:10.002071 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.002089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.003002 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.003005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.003972 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.003984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.004606 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.004610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.004721 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.005316 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:10.005333 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:10.008685 (+ 3352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":219,"threads_started":1}
W20260811 06:18:10.009140 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:10.000755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:40757 (local address 127.22.121.62:34539)
0811 06:18:10.000981 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:10.000987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:10.001186 (+ 199us) server_negotiation.cc:407] Connection header received
0811 06:18:10.001314 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:10.001318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:10.001394 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:10.001497 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:10.002199 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.002895 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.004098 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.004503 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.004605 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.008909 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:40757: BlockingRecv error: recv got EOF from 127.22.121.1:40757 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:10.363142 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:10.354854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:56135)
0811 06:18:10.355357 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:10.355373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:10.355500 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:10.355764 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:10.355767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:10.356164 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:10.356312 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.356321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.357263 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.357267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.358172 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.358184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.358793 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.358797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.358913 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.359503 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:10.359519 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:10.362803 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
W20260811 06:18:10.363205 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:10.354985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:56135 (local address 127.22.121.62:34539)
0811 06:18:10.355155 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:10.355160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:10.355434 (+ 274us) server_negotiation.cc:407] Connection header received
0811 06:18:10.355592 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:10.355595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:10.355657 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:10.355734 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:10.356426 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.357155 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.358280 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.358694 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.358779 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.362988 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:56135: BlockingRecv error: recv got EOF from 127.22.121.2:56135 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:10 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.3@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:10.748714 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:10.740539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.3:53889)
0811 06:18:10.740999 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:10.741019 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:10.741164 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:10.741524 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:10.741529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:10.742088 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:10.742289 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.742302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.743242 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.743245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.744156 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:10.744167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.744783 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.744787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:10.744905 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.745405 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:10.745422 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:10.748489 (+ 3067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":316,"threads_started":1}
W20260811 06:18:10.748872 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:10.740681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:53889 (local address 127.22.121.62:34539)
0811 06:18:10.740858 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:10.740865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:10.741114 (+ 249us) server_negotiation.cc:407] Connection header received
0811 06:18:10.741279 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:10.741284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:10.741367 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:10.741454 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:10.742419 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.743141 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.744279 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:10.744675 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:10.744766 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:10.748678 (+ 3912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.3:53889: BlockingRecv error: recv got EOF from 127.22.121.3:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 06:18:11 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:11 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.1@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:11.019027 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:11.010111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.1:41807)
0811 06:18:11.010573 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:11.010589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:11.010715 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:11.011012 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:11.011015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:11.011432 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:11.011634 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:11.011644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.013024 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.013028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:11.013962 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:11.013993 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.014686 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.014690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:11.014812 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.015377 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:11.015393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:11.018768 (+ 3375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
W20260811 06:18:11.019163 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:11.010216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:41807 (local address 127.22.121.62:34539)
0811 06:18:11.010437 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:11.010442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:11.010655 (+ 213us) server_negotiation.cc:407] Connection header received
0811 06:18:11.010805 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:11.010809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:11.010883 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:11.010960 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:11.011778 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.012867 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.014128 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.014579 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.014675 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.018939 (+ 4264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.1:41807: BlockingRecv error: recv got EOF from 127.22.121.1:41807 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 11 06:18:11 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:18:11 dist-test-slave-gjrz krb5kdc[5719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.121.2@KRBTEST.COM for oryx/127.22.121.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:18:11.373010 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:11.364380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:34539 (local address 127.22.121.2:38043)
0811 06:18:11.364829 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:11.364846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:11.365004 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:11.365559 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:11.365566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:11.365957 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:11.366149 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:11.366159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.367200 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.367204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:11.368148 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:11.368161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.368784 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.368788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:11.368919 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.369472 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:11.369490 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:11.372657 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.121.62:34539: Server oryx/127.22.121.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":216,"threads_started":1}
W20260811 06:18:11.373126 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:11.364436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:38043 (local address 127.22.121.62:34539)
0811 06:18:11.364606 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:11.364612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:11.364952 (+ 340us) server_negotiation.cc:407] Connection header received
0811 06:18:11.365131 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:11.365137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:11.365224 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:11.365327 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:11.366284 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.367070 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.368273 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.368678 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.368779 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.372880 (+ 4101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.121.2:38043: BlockingRecv error: recv got EOF from 127.22.121.2:38043 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:18:11.488174 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 6826
I20260811 06:18:11.502341 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:11.489190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.22.121.1:36499)
0811 06:18:11.489831 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:11.489838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:11.489872 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:18:11.489932 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:11.489939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:11.490196 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:11.490361 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:11.492161 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.493936 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.495421 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.496246 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.496369 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.499371 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:11.499385 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:11.499404 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:11.499454 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:11.501705 (+ 2251us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:18:11.502036 (+ 331us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53614: Request ticket server kudu/127.22.121.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":308,"threads_started":1}
W20260811 06:18:11.502429 7671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53614: Request ticket server kudu/127.22.121.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:18:11.502400 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 6826: Not authorized: Client connection negotiation failed: client connection to 127.22.121.1:36499: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:18:11.502583 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 6826
I20260811 06:18:11.526903 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7087
I20260811 06:18:11.538702 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:11.527897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.22.121.2:36207)
0811 06:18:11.528535 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:11.528542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:11.528573 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:18:11.528635 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:11.528641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:11.528884 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:11.529042 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:11.530614 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.531744 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.532901 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.533450 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.533543 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.535863 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:11.535872 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:11.535886 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:11.535922 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:11.538200 (+ 2278us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:18:11.538461 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50238: Request ticket server kudu/127.22.121.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":309,"threads_started":1}
W20260811 06:18:11.538776 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7087: Not authorized: Client connection negotiation failed: client connection to 127.22.121.2:36207: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:18:11.538806 7672 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50238: Request ticket server kudu/127.22.121.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:18:11.538928 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 7087
I20260811 06:18:11.561192 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7328
I20260811 06:18:11.572961 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:11.562200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.22.121.3:40455)
0811 06:18:11.562822 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:11.562829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:11.562854 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:18:11.562916 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:11.562923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:11.563170 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:11.563383 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:11.564660 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.565737 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.566978 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:11.567606 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:11.567726 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:11.569983 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:11.569992 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:11.570005 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:11.570056 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:11.572288 (+ 2232us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:18:11.572642 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38406: Request ticket server kudu/127.22.121.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":286,"threads_started":1}
W20260811 06:18:11.572928 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7328: Not authorized: Client connection negotiation failed: client connection to 127.22.121.3:40455: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.121.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:18:11.573067 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 7328
W20260811 06:18:11.573060 7673 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38406: Request ticket server kudu/127.22.121.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:18:11.592119 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 6683
I20260811 06:18:11.593243 6816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:11.704977 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 6683
2026-08-11T06:18:11Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21022 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T06:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:18:11Z Disabled control of system clock
I20260811 06:18:11.785971 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33559
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:33559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:18:12.096158 7684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:12.096400 7684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:12.096483 7684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:12.103335 7684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:18:12.103421 7684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:12.103466 7684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:12.103518 7684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:18:12.103559 7684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:18:12.112599 7684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:33559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:12.113684 7684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:12.115312 7684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:12.126334 7695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:12.126317 7698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:12.126575 7703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:12.126998 7684 server_base.cc:1034] running on GCE node
I20260811 06:18:12.127776 7684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:12.129032 7684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:12.130229 7684 hybrid_clock.cc:648] HybridClock initialized: now 1786429092130192 us; error 28 us; skew 500 ppm
I20260811 06:18:12.130573 7684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:12.132654 7684 webserver.cc:460] Webserver started at http://127.22.121.62:36851/ using document root <none> and password file <none>
I20260811 06:18:12.133285 7684 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:12.133378 7684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:12.133693 7684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:12.136724 7684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "a2a8369c478244f6af8b85f63b3cbbfa"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "4a38460811ca2f577600a4f8d53db451"
server_key_iv: "362f2382d2e0d00ed0cc3184b6e79c27"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.137470 7684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "a2a8369c478244f6af8b85f63b3cbbfa"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "4a38460811ca2f577600a4f8d53db451"
server_key_iv: "362f2382d2e0d00ed0cc3184b6e79c27"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.142225 7684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:18:12.145337 7711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:12.146541 7684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:18:12.146692 7684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a2a8369c478244f6af8b85f63b3cbbfa"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "4a38460811ca2f577600a4f8d53db451"
server_key_iv: "362f2382d2e0d00ed0cc3184b6e79c27"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.146873 7684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:12.174110 7684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:12.174819 7684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:12.175077 7684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:12.190806 7684 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:33559
I20260811 06:18:12.190832 7806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:33559 every 8 connection(s)
I20260811 06:18:12.191967 7684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:18:12.193311 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 7684
I20260811 06:18:12.193532 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:18:12.194337 23012 external_mini_cluster.cc:1428] Setting key 60126c223be0057d5c2a8ed2ff179e7b
I20260811 06:18:12.196362 7807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:12.208556 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:12.196339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.22.121.62:33559)
0811 06:18:12.197250 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:12.197259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:12.197299 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:18:12.198819 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:12.198838 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:12.199299 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:12.199620 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:12.200251 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.201840 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.203755 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.204730 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.204907 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:12.205948 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:12.205959 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:12.205977 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:12.206654 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:12.206750 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:12.206776 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:12.206908 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:12.207045 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:18:12.207401 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":325,"threads_started":1}
I20260811 06:18:12.209559 7807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa: Bootstrap starting.
I20260811 06:18:12.212399 7807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:12.213313 7807 log.cc:824] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa: Log is configured to *not* fsync() on all Append() calls
I20260811 06:18:12.215837 7807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa: No bootstrap required, opened a new log
I20260811 06:18:12.222702 7807 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } }
I20260811 06:18:12.223012 7807 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:12.223107 7807 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a8369c478244f6af8b85f63b3cbbfa, State: Initialized, Role: FOLLOWER
I20260811 06:18:12.223625 7807 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } }
I20260811 06:18:12.223842 7807 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:18:12.223963 7807 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:18:12.224116 7807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:12.226104 7807 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } }
I20260811 06:18:12.226599 7807 leader_election.cc:302] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a8369c478244f6af8b85f63b3cbbfa; no voters:
I20260811 06:18:12.227237 7807 leader_election.cc:288] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:18:12.227356 7814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:18:12.228018 7814 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [term 1 LEADER]: Becoming Leader. State: Replica: a2a8369c478244f6af8b85f63b3cbbfa, State: Running, Role: LEADER
I20260811 06:18:12.228551 7814 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } }
I20260811 06:18:12.229336 7807 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:18:12.231124 7817 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } } }
I20260811 06:18:12.231516 7817 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:12.232180 7822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a8369c478244f6af8b85f63b3cbbfa. Latest consensus state: current_term: 1 leader_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a8369c478244f6af8b85f63b3cbbfa" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33559 } } }
I20260811 06:18:12.232599 7822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:12.232853 7830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:18:12.237273 7830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:18:12.245378 7830 catalog_manager.cc:1347] Generated new cluster ID: 5cdd2d897efd4537b7e5f713882af494
I20260811 06:18:12.245467 7830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:18:12.268949 7830 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:18:12.270634 7830 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:18:12.290122 7830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a8369c478244f6af8b85f63b3cbbfa: Generated new TSK 0
I20260811 06:18:12.291108 7830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:18:12.308590 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--builtin_ntp_servers=127.22.121.20:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 06:18:12.622605 7850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:12.622852 7850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:12.622943 7850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:12.629729 7850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:12.629812 7850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:12.629918 7850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:18:12.638453 7850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:12.639506 7850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:12.641112 7850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:12.652957 7860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:12.652997 7862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:12.653127 7859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:12.653642 7850 server_base.cc:1034] running on GCE node
I20260811 06:18:12.654176 7850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:12.654940 7850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:12.656123 7850 hybrid_clock.cc:648] HybridClock initialized: now 1786429092656065 us; error 47 us; skew 500 ppm
I20260811 06:18:12.656378 7850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:12.658339 7850 webserver.cc:460] Webserver started at http://127.22.121.1:44445/ using document root <none> and password file <none>
I20260811 06:18:12.658917 7850 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:12.658998 7850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:12.659256 7850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:12.661819 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "f289404a3b8d46998bbc436ae6bb459d"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "3c6b845129c58b622835a0ca5f320ece"
server_key_iv: "544857bc67ed277ec54fc17c3de92e82"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.662525 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "f289404a3b8d46998bbc436ae6bb459d"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "3c6b845129c58b622835a0ca5f320ece"
server_key_iv: "544857bc67ed277ec54fc17c3de92e82"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.667017 7850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:18:12.670095 7874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:12.671250 7850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:18:12.671416 7850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "f289404a3b8d46998bbc436ae6bb459d"
format_stamp: "Formatted at 2026-08-11 06:18:12 on dist-test-slave-gjrz"
server_key: "3c6b845129c58b622835a0ca5f320ece"
server_key_iv: "544857bc67ed277ec54fc17c3de92e82"
server_key_version: "encryptionkey@0"
I20260811 06:18:12.671577 7850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:12.685607 7850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:12.686255 7850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:12.686484 7850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:12.687235 7850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:12.688455 7850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:12.688540 7850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:12.688616 7850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:12.688669 7850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:12.717264 7850 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:32841
I20260811 06:18:12.717324 8073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:32841 every 8 connection(s)
I20260811 06:18:12.718338 7850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:18:12.721132 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 7850
I20260811 06:18:12.721318 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:18:12.722023 23012 external_mini_cluster.cc:1428] Setting key 1641ae7b03efa148021f8ae0751824e4
I20260811 06:18:12.724702 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--builtin_ntp_servers=127.22.121.20:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:18:12.733850 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:12.720497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:56029 (local address 127.22.121.62:33559)
0811 06:18:12.720999 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:12.721006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:12.722158 (+ 1152us) server_negotiation.cc:407] Connection header received
0811 06:18:12.724228 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:12.724233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:12.724308 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:12.724398 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:12.725426 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.726129 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.727631 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.728017 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.728134 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:12.733097 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:12.733104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:12.733109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:12.733129 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:12.733188 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:12.733204 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:12.733326 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:12.733532 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 06:18:12.733610 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260811 06:18:12.735100 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:12.721011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33559 (local address 127.22.121.1:56029)
0811 06:18:12.722013 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:12.722055 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:12.723986 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:12.724635 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:12.724643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:12.724722 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:12.725274 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:12.725290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.726271 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.726275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:12.727497 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:12.727509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:12.728226 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:12.728234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:12.728443 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:12.732720 (+ 4277us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:12.732724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:12.732772 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:12.732874 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:12.733325 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:12.733333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:12.733339 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:18:12.733506 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 06:18:12.733863 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":318,"threads_started":1}
I20260811 06:18:12.736579 8074 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33559
I20260811 06:18:12.736865 8074 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:12.737696 8074 heartbeater.cc:511] Master 127.22.121.62:33559 requested a full tablet report, sending...
I20260811 06:18:12.739710 7743 ts_manager.cc:194] Registered new tserver with Master: f289404a3b8d46998bbc436ae6bb459d (127.22.121.1:32841)
I20260811 06:18:12.741091 7743 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.1:56029
W20260811 06:18:13.041508 8084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:13.041754 8084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:13.041870 8084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:13.048949 8084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:13.049041 8084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:13.049177 8084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:18:13.057729 8084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:13.058889 8084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:13.060556 8084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:13.072436 8098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:13.072539 8095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:13.072500 8096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:13.073364 8084 server_base.cc:1034] running on GCE node
I20260811 06:18:13.073927 8084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:13.074818 8084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:13.076022 8084 hybrid_clock.cc:648] HybridClock initialized: now 1786429093075950 us; error 61 us; skew 500 ppm
I20260811 06:18:13.076303 8084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:13.078292 8084 webserver.cc:460] Webserver started at http://127.22.121.2:42171/ using document root <none> and password file <none>
I20260811 06:18:13.078908 8084 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:13.079012 8084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:13.079337 8084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:13.081902 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "ca3ff475c08746f7b0cc74a754d98776"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "f53f8c54d7d50bf4b49ea54b72777098"
server_key_iv: "ca5ad29d7478ebf39d600aacf193bb3e"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.082646 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "ca3ff475c08746f7b0cc74a754d98776"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "f53f8c54d7d50bf4b49ea54b72777098"
server_key_iv: "ca5ad29d7478ebf39d600aacf193bb3e"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.087246 8084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 06:18:13.090349 8105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.091531 8084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:18:13.091675 8084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "ca3ff475c08746f7b0cc74a754d98776"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "f53f8c54d7d50bf4b49ea54b72777098"
server_key_iv: "ca5ad29d7478ebf39d600aacf193bb3e"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.091857 8084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:13.107400 8084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:13.108114 8084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:13.108377 8084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:13.109215 8084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:13.110493 8084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:13.110586 8084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.110694 8084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:13.110766 8084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.139926 8084 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:40065
I20260811 06:18:13.139971 8329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:40065 every 8 connection(s)
I20260811 06:18:13.141081 8084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:18:13.149817 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 8084
I20260811 06:18:13.150027 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:18:13.150579 23012 external_mini_cluster.cc:1428] Setting key df15a67efdff21de9eb48f61585d5ab2
I20260811 06:18:13.153045 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--builtin_ntp_servers=127.22.121.20:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:18:13.156740 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.143346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:41783 (local address 127.22.121.62:33559)
0811 06:18:13.143546 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.143552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.144867 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 06:18:13.146501 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.146505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.146572 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.146657 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.147619 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.148325 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.149810 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.150418 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.150550 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.155994 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:13.156000 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:13.156004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:13.156023 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:13.156057 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:13.156070 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:13.156121 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:13.156378 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:18:13.156466 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:18:13.158206 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.143870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33559 (local address 127.22.121.2:41783)
0811 06:18:13.144712 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:13.144750 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:13.146236 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:13.146832 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:13.146839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:13.146920 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:13.147482 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:13.147500 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.148447 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.148450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:13.149675 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:13.149687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.150626 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.150634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:13.150857 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.155680 (+ 4823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:13.155683 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:13.155726 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:13.155803 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:13.156194 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:13.156203 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:13.156209 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:18:13.156900 (+ 691us) client_negotiation.cc:240] Negotiation successful
0811 06:18:13.157213 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":274,"threads_started":1}
I20260811 06:18:13.159631 8331 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33559
I20260811 06:18:13.159883 8331 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:13.160564 8331 heartbeater.cc:511] Master 127.22.121.62:33559 requested a full tablet report, sending...
I20260811 06:18:13.161906 7743 ts_manager.cc:194] Registered new tserver with Master: ca3ff475c08746f7b0cc74a754d98776 (127.22.121.2:40065)
I20260811 06:18:13.162901 7743 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.2:41783
W20260811 06:18:13.480203 8335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:13.480437 8335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:13.480528 8335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:13.487349 8335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:13.487430 8335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:13.487535 8335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:18:13.496882 8335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:13.498241 8335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:13.500350 8335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:13.512612 8345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:13.512573 8346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:13.512674 8350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:13.513367 8335 server_base.cc:1034] running on GCE node
I20260811 06:18:13.513885 8335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:13.514667 8335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:13.515851 8335 hybrid_clock.cc:648] HybridClock initialized: now 1786429093515805 us; error 38 us; skew 500 ppm
I20260811 06:18:13.516120 8335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:13.518054 8335 webserver.cc:460] Webserver started at http://127.22.121.3:40357/ using document root <none> and password file <none>
I20260811 06:18:13.518639 8335 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:13.518724 8335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:13.518976 8335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:13.521596 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "b9f9a6ea10b446b6a80206d811835cdd"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "8127ba7b0c80463b02a3cf3fd8e7e481"
server_key_iv: "abf325a9746690e326d984998165f35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.522313 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "b9f9a6ea10b446b6a80206d811835cdd"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "8127ba7b0c80463b02a3cf3fd8e7e481"
server_key_iv: "abf325a9746690e326d984998165f35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.526886 8335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:18:13.530416 8360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.531641 8335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:18:13.531766 8335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "b9f9a6ea10b446b6a80206d811835cdd"
format_stamp: "Formatted at 2026-08-11 06:18:13 on dist-test-slave-gjrz"
server_key: "8127ba7b0c80463b02a3cf3fd8e7e481"
server_key_iv: "abf325a9746690e326d984998165f35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:13.531927 8335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:13.549628 8335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:13.550339 8335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:13.550575 8335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:13.551450 8335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:13.552759 8335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:13.552847 8335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.552919 8335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:13.552980 8335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:13.582304 8335 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41225
I20260811 06:18:13.582367 8586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41225 every 8 connection(s)
I20260811 06:18:13.583592 8335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:18:13.590615 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 8335
I20260811 06:18:13.590776 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:18:13.591192 23012 external_mini_cluster.cc:1428] Setting key ab0d905126aa6c112889e515f2cdceab
I20260811 06:18:13.595340 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.585994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:57703 (local address 127.22.121.62:33559)
0811 06:18:13.586182 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.586187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.587531 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 06:18:13.589159 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.589163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.589228 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.589312 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.590271 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.591063 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.592799 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.593281 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.593423 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.594666 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:13.594671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:13.594674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:13.594690 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:13.594720 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:13.594732 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:13.594800 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:13.594983 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 06:18:13.595079 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:18:13.596298 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.586476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33559 (local address 127.22.121.3:57703)
0811 06:18:13.587390 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:13.587433 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:13.588933 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:13.589489 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:13.589496 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:13.589577 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:13.590127 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:13.590143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.591215 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.591219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:13.592667 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:13.592680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.593489 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.593495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:13.593696 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.594359 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:13.594362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:13.594399 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:13.594507 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:13.594814 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:13.594824 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:13.594832 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:18:13.594972 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 06:18:13.595254 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":259,"threads_started":1}
I20260811 06:18:13.597486 8588 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33559
I20260811 06:18:13.597738 8588 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:13.598407 8588 heartbeater.cc:511] Master 127.22.121.62:33559 requested a full tablet report, sending...
I20260811 06:18:13.599658 7743 ts_manager.cc:194] Registered new tserver with Master: b9f9a6ea10b446b6a80206d811835cdd (127.22.121.3:41225)
I20260811 06:18:13.600382 7743 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.3:57703
I20260811 06:18:13.606096 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:18:13.611814 8078 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:13.610340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.22.121.62:33559)
0811 06:18:13.610502 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.610506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.610996 (+ 490us) server_negotiation.cc:407] Connection header received
0811 06:18:13.611135 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.611138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.611208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.611308 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.611606 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51202: BlockingRecv error: recv got EOF from 127.0.0.1:51202 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:18:13.612690 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7850
I20260811 06:18:13.619490 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.613561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.22.121.1:32841)
0811 06:18:13.614023 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.614029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.614054 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:18:13.614115 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.614120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.614293 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.614405 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.614727 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.615699 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.616826 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.617332 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.617424 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.618269 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:13.618276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:13.618286 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:13.618813 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:13.618856 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:13.618874 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:13.618964 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:13.619036 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 06:18:13.619135 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":229,"threads_started":1}
I20260811 06:18:13.620595 8066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:13.700062 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 7850
I20260811 06:18:13.723977 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 8084
I20260811 06:18:13.731253 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.724800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.22.121.2:40065)
0811 06:18:13.725309 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.725316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.725345 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:18:13.725414 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.725421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.725641 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.725775 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.726085 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.727079 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.728289 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.728848 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.728942 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.729867 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:13.729877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:13.729891 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:13.730522 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:13.730579 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:13.730600 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:13.730687 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:13.730798 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 06:18:13.730941 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":269,"threads_started":1}
I20260811 06:18:13.732481 8323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:13.810842 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 8084
I20260811 06:18:13.833861 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 8335
I20260811 06:18:13.841869 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:13.834764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.22.121.3:41225)
0811 06:18:13.835506 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:13.835513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:13.835544 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:18:13.835607 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:13.835613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:13.835857 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:13.836023 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:13.836510 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.837558 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.838779 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:13.839379 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:13.839490 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:13.840482 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:13.840491 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:13.840505 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:13.841065 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:13.841121 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:13.841141 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:13.841242 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:13.841391 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 06:18:13.841531 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":395,"threads_started":1}
I20260811 06:18:13.843076 8581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:13.922688 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 8335
I20260811 06:18:13.944528 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 7684
I20260811 06:18:13.947036 7798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:14.046020 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 7684
2026-08-11T06:18:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2334 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T06:18:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:18:14Z Disabled control of system clock
I20260811 06:18:14.121397 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46255
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:18:14.433559 8631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:14.433816 8631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:14.433957 8631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:14.440918 8631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:18:14.441016 8631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:14.441087 8631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:14.441166 8631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:18:14.441226 8631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:18:14.450067 8631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:14.451212 8631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:14.452910 8631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:14.463850 8640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:14.463893 8646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:14.463805 8644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:14.464934 8631 server_base.cc:1034] running on GCE node
I20260811 06:18:14.465636 8631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:14.466882 8631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:14.468072 8631 hybrid_clock.cc:648] HybridClock initialized: now 1786429094468023 us; error 40 us; skew 500 ppm
I20260811 06:18:14.468370 8631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:14.470400 8631 webserver.cc:460] Webserver started at http://127.22.121.62:39791/ using document root <none> and password file <none>
I20260811 06:18:14.471030 8631 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:14.471130 8631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:14.471448 8631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:14.474097 8631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "e72a96cfa2c24ba289cae8104279b6f8"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "e964d6abe6fc9496c1cc2e2d4e3e77b1"
server_key_iv: "9d040c36f4a77c27cd5a70208cb127eb"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.474845 8631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "e72a96cfa2c24ba289cae8104279b6f8"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "e964d6abe6fc9496c1cc2e2d4e3e77b1"
server_key_iv: "9d040c36f4a77c27cd5a70208cb127eb"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.479497 8631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260811 06:18:14.482571 8657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:14.483795 8631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:18:14.483945 8631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "e72a96cfa2c24ba289cae8104279b6f8"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "e964d6abe6fc9496c1cc2e2d4e3e77b1"
server_key_iv: "9d040c36f4a77c27cd5a70208cb127eb"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.484128 8631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:14.489468 8631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:14.490132 8631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:14.490383 8631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:14.506047 8631 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46255
I20260811 06:18:14.506079 8752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46255 every 8 connection(s)
I20260811 06:18:14.507157 8631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:18:14.508526 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 8631
I20260811 06:18:14.508704 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:18:14.509313 23012 external_mini_cluster.cc:1428] Setting key c34efc81ccd6bebcebe6040764145d9b
I20260811 06:18:14.510972 8755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:14.517786 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:14.511128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.22.121.62:46255)
0811 06:18:14.512200 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:14.512213 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:14.512263 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:18:14.514008 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:14.514034 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:14.514552 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:14.514955 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:14.515938 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:14.515947 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:14.515963 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:14.515978 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:14.516058 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:14.516092 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:14.516305 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:14.516440 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 06:18:14.516682 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":320,"threads_started":1}
I20260811 06:18:14.523238 8755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8: Bootstrap starting.
I20260811 06:18:14.525981 8755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:14.526957 8755 log.cc:824] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:18:14.529253 8755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8: No bootstrap required, opened a new log
I20260811 06:18:14.536168 8755 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } }
I20260811 06:18:14.536474 8755 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:14.536564 8755 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72a96cfa2c24ba289cae8104279b6f8, State: Initialized, Role: FOLLOWER
I20260811 06:18:14.537043 8755 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } }
I20260811 06:18:14.537257 8755 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:18:14.537379 8755 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:18:14.537529 8755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:14.539559 8755 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } }
I20260811 06:18:14.540002 8755 leader_election.cc:302] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72a96cfa2c24ba289cae8104279b6f8; no voters:
I20260811 06:18:14.540498 8755 leader_election.cc:288] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:18:14.540603 8762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:18:14.541225 8762 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [term 1 LEADER]: Becoming Leader. State: Replica: e72a96cfa2c24ba289cae8104279b6f8, State: Running, Role: LEADER
I20260811 06:18:14.541752 8762 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } }
I20260811 06:18:14.542502 8755 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:18:14.544584 8765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72a96cfa2c24ba289cae8104279b6f8. Latest consensus state: current_term: 1 leader_uuid: "e72a96cfa2c24ba289cae8104279b6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } } }
I20260811 06:18:14.544946 8765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:14.547665 8764 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e72a96cfa2c24ba289cae8104279b6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72a96cfa2c24ba289cae8104279b6f8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46255 } } }
I20260811 06:18:14.548090 8764 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:14.548427 8776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:18:14.553013 8776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:18:14.561378 8776 catalog_manager.cc:1347] Generated new cluster ID: 094216a650c84a5da8ac00774a687b3a
I20260811 06:18:14.561502 8776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:18:14.582726 8776 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:18:14.583073 8776 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:18:14.589144 8776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e72a96cfa2c24ba289cae8104279b6f8: Generated new TSK 0
I20260811 06:18:14.589824 8776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:18:14.600247 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--builtin_ntp_servers=127.22.121.20:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:18:14.917857 8793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:14.918123 8793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:14.918257 8793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:14.925172 8793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:14.925266 8793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:14.925410 8793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:18:14.933903 8793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:14.935009 8793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:14.936710 8793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:14.948961 8811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:14.949064 8810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:14.948997 8816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:14.949813 8793 server_base.cc:1034] running on GCE node
I20260811 06:18:14.950395 8793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:14.951207 8793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:14.952431 8793 hybrid_clock.cc:648] HybridClock initialized: now 1786429094952390 us; error 33 us; skew 500 ppm
I20260811 06:18:14.952720 8793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:14.954725 8793 webserver.cc:460] Webserver started at http://127.22.121.1:45099/ using document root <none> and password file <none>
I20260811 06:18:14.955389 8793 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:14.955478 8793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:14.955752 8793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:14.958415 8793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "92e7a33bfb474981b58788ca095e6599"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "10c64525cec644c82df351de856ae034"
server_key_iv: "2f3056602946dc29435ac968d246a35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.959192 8793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "92e7a33bfb474981b58788ca095e6599"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "10c64525cec644c82df351de856ae034"
server_key_iv: "2f3056602946dc29435ac968d246a35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.964116 8793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:18:14.967232 8827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:14.968447 8793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:18:14.968591 8793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "92e7a33bfb474981b58788ca095e6599"
format_stamp: "Formatted at 2026-08-11 06:18:14 on dist-test-slave-gjrz"
server_key: "10c64525cec644c82df351de856ae034"
server_key_iv: "2f3056602946dc29435ac968d246a35b"
server_key_version: "encryptionkey@0"
I20260811 06:18:14.968773 8793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:14.973995 8793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:14.974689 8793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:14.974920 8793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:14.975855 8793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:14.977159 8793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:14.977252 8793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:14.977351 8793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:14.977423 8793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.006696 8793 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:35245
I20260811 06:18:15.006755 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:35245 every 8 connection(s)
I20260811 06:18:15.007917 8793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:18:15.015008 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 8793
I20260811 06:18:15.015205 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:18:15.015317 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.010248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:51893 (local address 127.22.121.62:46255)
0811 06:18:15.010456 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.010462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.011777 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 06:18:15.013349 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.013353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.013405 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.013490 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.014611 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:15.014618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:15.014622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:15.014643 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:15.014685 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:15.014702 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:15.014825 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:15.014951 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:18:15.015016 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:18:15.015952 23012 external_mini_cluster.cc:1428] Setting key 3aec6f0fe4ec6ee207d97bf4af40ca1e
I20260811 06:18:15.016155 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.010739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46255 (local address 127.22.121.1:51893)
0811 06:18:15.011641 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:15.011678 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:15.013116 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:15.013664 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:15.013672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:15.013755 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:15.014322 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:15.014325 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:15.014360 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:15.014440 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:15.014790 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:15.014796 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:15.014801 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:18:15.014990 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 06:18:15.015212 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":250,"threads_started":1}
I20260811 06:18:15.017639 9057 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46255
I20260811 06:18:15.017894 9057 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:15.018528 9057 heartbeater.cc:511] Master 127.22.121.62:46255 requested a full tablet report, sending...
I20260811 06:18:15.019232 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--builtin_ntp_servers=127.22.121.20:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:18:15.020776 8679 ts_manager.cc:194] Registered new tserver with Master: 92e7a33bfb474981b58788ca095e6599 (127.22.121.1:35245)
W20260811 06:18:15.332250 9061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:15.332492 9061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:15.332619 9061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:15.339418 9061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:15.339501 9061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:15.339658 9061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:18:15.348333 9061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:15.349439 9061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:15.351087 9061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:15.363363 9075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:15.363442 9073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:15.363401 9081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:15.364459 9061 server_base.cc:1034] running on GCE node
I20260811 06:18:15.364995 9061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:15.365824 9061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:15.367019 9061 hybrid_clock.cc:648] HybridClock initialized: now 1786429095366968 us; error 41 us; skew 500 ppm
I20260811 06:18:15.367347 9061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:15.369345 9061 webserver.cc:460] Webserver started at http://127.22.121.2:40415/ using document root <none> and password file <none>
I20260811 06:18:15.369956 9061 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:15.370057 9061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:15.370360 9061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:15.375711 9061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "595b4623851f4974a1a616cb1068a11f"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "dd5a99eb1401a10aa7e87a18cce2c4f7"
server_key_iv: "a10bd5e37cfe8265b657ecec2cfc0fde"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.376463 9061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "595b4623851f4974a1a616cb1068a11f"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "dd5a99eb1401a10aa7e87a18cce2c4f7"
server_key_iv: "a10bd5e37cfe8265b657ecec2cfc0fde"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.381228 9061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:18:15.384370 9088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.385568 9061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:18:15.385715 9061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "595b4623851f4974a1a616cb1068a11f"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "dd5a99eb1401a10aa7e87a18cce2c4f7"
server_key_iv: "a10bd5e37cfe8265b657ecec2cfc0fde"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.385906 9061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:15.391252 9061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:15.392033 9061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:15.392278 9061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:15.393101 9061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:15.394374 9061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:15.394467 9061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.394573 9061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:15.394642 9061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.424036 9061 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:37871
I20260811 06:18:15.424124 9316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:37871 every 8 connection(s)
I20260811 06:18:15.425206 9061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:18:15.431137 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 9061
I20260811 06:18:15.431311 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:18:15.431752 23012 external_mini_cluster.cc:1428] Setting key f770b3c13e2b8b208dc25032e6c8eedd
I20260811 06:18:15.432881 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.427550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:43977 (local address 127.22.121.62:46255)
0811 06:18:15.427729 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.427734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.429062 (+ 1328us) server_negotiation.cc:407] Connection header received
0811 06:18:15.430669 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.430675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.430741 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.430831 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.432108 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:15.432114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:15.432119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:15.432138 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:15.432169 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:15.432181 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:15.432308 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:15.432509 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 06:18:15.432583 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:18:15.433760 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.428029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46255 (local address 127.22.121.2:43977)
0811 06:18:15.428915 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:15.428955 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:15.430408 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:15.431078 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:15.431085 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:15.431169 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:15.431799 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:15.431801 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:15.431840 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:15.431945 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:15.432290 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:15.432299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:15.432306 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:18:15.432560 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 06:18:15.432806 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":295,"threads_started":1}
I20260811 06:18:15.434604 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--builtin_ntp_servers=127.22.121.20:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:18:15.435132 9319 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46255
I20260811 06:18:15.435420 9319 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:15.435951 9319 heartbeater.cc:511] Master 127.22.121.62:46255 requested a full tablet report, sending...
I20260811 06:18:15.437222 8679 ts_manager.cc:194] Registered new tserver with Master: 595b4623851f4974a1a616cb1068a11f (127.22.121.2:37871)
W20260811 06:18:15.750556 9325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:15.750819 9325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:15.750950 9325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:15.757840 9325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:15.757932 9325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:15.758073 9325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:18:15.766763 9325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:15.767927 9325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:15.769739 9325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:15.781843 9339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:15.781941 9343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:15.781962 9345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:15.782943 9325 server_base.cc:1034] running on GCE node
I20260811 06:18:15.783473 9325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:15.784256 9325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:15.785440 9325 hybrid_clock.cc:648] HybridClock initialized: now 1786429095785396 us; error 38 us; skew 500 ppm
I20260811 06:18:15.785697 9325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:18:15.787725 9325 webserver.cc:460] Webserver started at http://127.22.121.3:40175/ using document root <none> and password file <none>
I20260811 06:18:15.788314 9325 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:15.788424 9325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:15.788723 9325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:15.791400 9325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "6b17d4e9ba5046779320823ee24afbad"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "8f2d71f561845038f8d1147f83ed7926"
server_key_iv: "0e6fadc269b662da02257f8943c8f6b0"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.792115 9325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "6b17d4e9ba5046779320823ee24afbad"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "8f2d71f561845038f8d1147f83ed7926"
server_key_iv: "0e6fadc269b662da02257f8943c8f6b0"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.796700 9325 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:18:15.799769 9361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.800904 9325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:18:15.801036 9325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "6b17d4e9ba5046779320823ee24afbad"
format_stamp: "Formatted at 2026-08-11 06:18:15 on dist-test-slave-gjrz"
server_key: "8f2d71f561845038f8d1147f83ed7926"
server_key_iv: "0e6fadc269b662da02257f8943c8f6b0"
server_key_version: "encryptionkey@0"
I20260811 06:18:15.801234 9325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:15.806325 9325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:15.806993 9325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:15.807226 9325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:15.808063 9325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:15.809300 9325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:15.809387 9325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.809490 9325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:15.809561 9325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:15.838518 9325 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:39255
I20260811 06:18:15.838587 9569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:39255 every 8 connection(s)
I20260811 06:18:15.839684 9325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:18:15.847069 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 9325
I20260811 06:18:15.847126 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.841922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:55823 (local address 127.22.121.62:46255)
0811 06:18:15.842094 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.842099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.843410 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 06:18:15.845026 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.845032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.845099 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.845186 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.846394 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:15.846400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:15.846405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:15.846424 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:15.846457 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:15.846469 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:15.846591 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:15.846783 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:18:15.846854 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:18:15.847235 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:18:15.847679 23012 external_mini_cluster.cc:1428] Setting key a5075bdf4bae7a12d2fb3e55a9c7530c
I20260811 06:18:15.847975 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.842346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46255 (local address 127.22.121.3:55823)
0811 06:18:15.843211 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:15.843309 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:15.844775 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:15.845413 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:15.845421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:15.845506 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:15.846110 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:18:15.846112 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:15.846148 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:18:15.846229 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:15.846555 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:15.846567 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:15.846575 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:18:15.846829 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 06:18:15.847052 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":258,"threads_started":1}
I20260811 06:18:15.849238 9571 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46255
I20260811 06:18:15.849542 9571 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:15.850145 9571 heartbeater.cc:511] Master 127.22.121.62:46255 requested a full tablet report, sending...
I20260811 06:18:15.851521 8683 ts_manager.cc:194] Registered new tserver with Master: 6b17d4e9ba5046779320823ee24afbad (127.22.121.3:39255)
I20260811 06:18:15.862398 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:18:15.868377 8758 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:18:15.866826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.22.121.62:46255)
0811 06:18:15.866992 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.866996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.867545 (+ 549us) server_negotiation.cc:407] Connection header received
0811 06:18:15.867695 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.867698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.867741 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.867819 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.868180 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59412: BlockingRecv error: recv got EOF from 127.0.0.1:59412 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:18:15.868942 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 8793
I20260811 06:18:15.872097 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.869851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.22.121.1:35245)
0811 06:18:15.870314 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.870320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.870343 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:18:15.870398 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.870403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.870561 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.870683 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.871473 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:15.871479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:15.871488 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:15.871506 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:15.871542 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:15.871555 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:15.871661 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:15.871741 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:18:15.871818 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":205,"threads_started":1}
I20260811 06:18:15.873183 9046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:15.951563 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 8793
I20260811 06:18:15.976624 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 9061
I20260811 06:18:15.979918 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:15.977535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.22.121.2:37871)
0811 06:18:15.978007 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:15.978014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:15.978040 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:18:15.978101 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:15.978107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:15.978313 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:15.978432 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:15.979307 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:15.979315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:15.979327 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:15.979347 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:15.979391 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:15.979406 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:15.979507 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:15.979593 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 06:18:15.979675 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":208,"threads_started":1}
I20260811 06:18:15.981104 9305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:16.060161 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 9061
I20260811 06:18:16.081506 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 9325
I20260811 06:18:16.085493 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:16.082460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.22.121.3:39255)
0811 06:18:16.083118 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:16.083125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:16.083154 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:18:16.083216 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:16.083222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:16.083482 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:16.083628 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:16.084734 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:16.084741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:16.084752 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:18:16.084768 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:16.084807 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:18:16.084821 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:16.084972 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:16.085101 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:18:16.085202 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":346,"threads_started":1}
I20260811 06:18:16.086699 9564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:16.166188 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 9325
I20260811 06:18:16.186818 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 8631
I20260811 06:18:16.187992 8743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:16.289885 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 8631
2026-08-11T06:18:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2245 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:18:16 dist-test-slave-gjrz krb5kdc[9606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:18:16 dist-test-slave-gjrz krb5kdc[9606](info): set up 2 sockets
Aug 11 06:18:16 dist-test-slave-gjrz krb5kdc[9606](info): commencing operation
krb5kdc: starting...
W20260811 06:18:17.534014 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:18:17 dist-test-slave-gjrz krb5kdc[9606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:18:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:18:17.746804 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35451
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:35451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:18:18.057945 9630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:18.058212 9630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:18.058339 9630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:18.065634 9630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:18:18.065728 9630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:18.065801 9630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:18.065878 9630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:18:18.065936 9630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:18:18.075212 9630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:35451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:35451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:18.076435 9630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:18.077672 9630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:18.088797 9638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:18.088912 9639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:18.088872 9648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:18.089756 9630 server_base.cc:1034] running on GCE node
I20260811 06:18:18.090476 9630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:18.091790 9630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:18.093007 9630 hybrid_clock.cc:648] HybridClock initialized: now 1786429098092961 us; error 29 us; skew 500 ppm
I20260811 06:18:18.093323 9630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:18:18 dist-test-slave-gjrz krb5kdc[9606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429098, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:18:18.099552 9630 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:18:18.100867 9630 webserver.cc:460] Webserver started at http://127.22.121.62:39553/ using document root <none> and password file <none>
I20260811 06:18:18.101481 9630 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:18.101608 9630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:18.101908 9630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:18.104828 9630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "8091ccf5cb684c36a15dfafc96cf082f"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "f9001949f6e15ffeffd3c810abace619"
server_key_iv: "cebcbad92653c3ee04da01e5d1af1636"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.105545 9630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "8091ccf5cb684c36a15dfafc96cf082f"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "f9001949f6e15ffeffd3c810abace619"
server_key_iv: "cebcbad92653c3ee04da01e5d1af1636"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.110286 9630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:18:18.113427 9664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:18.114599 9630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:18:18.114761 9630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "8091ccf5cb684c36a15dfafc96cf082f"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "f9001949f6e15ffeffd3c810abace619"
server_key_iv: "cebcbad92653c3ee04da01e5d1af1636"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.114931 9630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:18.145663 9630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:18.149272 9630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:18.149528 9630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:18.166106 9630 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:35451
I20260811 06:18:18.166139 9748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:35451 every 8 connection(s)
I20260811 06:18:18.167472 9630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:18:18.170957 9749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:18.173559 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 9630
I20260811 06:18:18.173755 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:18:18.174388 23012 external_mini_cluster.cc:1428] Setting key d32a3363dccb75d4d5f9e23a8186cc33
I20260811 06:18:18.181850 9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: Bootstrap starting.
I20260811 06:18:18.184718 9749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:18.185708 9749 log.cc:824] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:18:18 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:18:18.188611 9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: No bootstrap required, opened a new log
I20260811 06:18:18.195000 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:18.176323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.22.121.62:35451)
0811 06:18:18.177050 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:18.177062 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:18.177102 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:18:18.178699 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:18.178720 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:18.179077 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:18.179464 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:18.180897 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.182184 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.183498 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.184133 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.184305 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:18.188322 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:18.188330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:18.188343 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:18.188388 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:18.191233 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:18.192017 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:18.192023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:18.192029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:18.192200 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:18.192577 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:18.192581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:18.192583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:18.193088 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:18.193337 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:18.193716 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 06:18:18.194039 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":318,"threads_started":1}
I20260811 06:18:18.196947 9749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } }
I20260811 06:18:18.197343 9749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:18.197454 9749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8091ccf5cb684c36a15dfafc96cf082f, State: Initialized, Role: FOLLOWER
I20260811 06:18:18.198024 9749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } }
I20260811 06:18:18.198230 9749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:18:18.198325 9749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:18:18.198438 9749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:18.200517 9749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } }
I20260811 06:18:18.201042 9749 leader_election.cc:302] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8091ccf5cb684c36a15dfafc96cf082f; no voters:
I20260811 06:18:18.201591 9749 leader_election.cc:288] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:18:18.201681 9759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:18:18.202329 9759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [term 1 LEADER]: Becoming Leader. State: Replica: 8091ccf5cb684c36a15dfafc96cf082f, State: Running, Role: LEADER
I20260811 06:18:18.202838 9759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } }
I20260811 06:18:18.203410 9749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:18:18.205508 9762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8091ccf5cb684c36a15dfafc96cf082f. Latest consensus state: current_term: 1 leader_uuid: "8091ccf5cb684c36a15dfafc96cf082f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } } }
I20260811 06:18:18.205889 9762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:18.205571 9760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8091ccf5cb684c36a15dfafc96cf082f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091ccf5cb684c36a15dfafc96cf082f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 35451 } } }
I20260811 06:18:18.206125 9760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f [sys.catalog]: This master's current role is: LEADER
I20260811 06:18:18.209565 9769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:18:18.214143 9769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:18:18.224315 9769 catalog_manager.cc:1347] Generated new cluster ID: a6ad9c53dfa046628698d9a09538a05a
I20260811 06:18:18.224424 9769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:18:18.261293 9769 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:18:18.262981 9769 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:18:18.273002 9769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: Generated new TSK 0
I20260811 06:18:18.273717 9769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:18:18.342546 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:18:18.655546 9801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:18.655789 9801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:18.655885 9801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:18.662671 9801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:18.662758 9801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:18.662868 9801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:18:18.671923 9801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:18.672967 9801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:18.674108 9801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:18.687676 9812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:18.687731 9809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:18.687733 9817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:18.688452 9801 server_base.cc:1034] running on GCE node
I20260811 06:18:18.688988 9801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:18.689859 9801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:18.691054 9801 hybrid_clock.cc:648] HybridClock initialized: now 1786429098691012 us; error 38 us; skew 500 ppm
I20260811 06:18:18.691370 9801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:18:18 dist-test-slave-gjrz krb5kdc[9606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429098, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:18:18.694183 9801 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:18:18.695572 9801 webserver.cc:460] Webserver started at http://127.22.121.1:41443/ using document root <none> and password file <none>
I20260811 06:18:18.696198 9801 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:18.696302 9801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:18.696574 9801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:18.699198 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "cf57a69260f64a24b9fa441c560517c9"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "dc6ecdebd7a29e0c1b820b11ea338c6b"
server_key_iv: "5177e5f4b3dc1a52cd6465329a5147f5"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.699991 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "cf57a69260f64a24b9fa441c560517c9"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "dc6ecdebd7a29e0c1b820b11ea338c6b"
server_key_iv: "5177e5f4b3dc1a52cd6465329a5147f5"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.704879 9801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:18:18.708130 9826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:18.709306 9801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:18:18.709447 9801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "cf57a69260f64a24b9fa441c560517c9"
format_stamp: "Formatted at 2026-08-11 06:18:18 on dist-test-slave-gjrz"
server_key: "dc6ecdebd7a29e0c1b820b11ea338c6b"
server_key_iv: "5177e5f4b3dc1a52cd6465329a5147f5"
server_key_version: "encryptionkey@0"
I20260811 06:18:18.709626 9801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:18.735823 9801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:18.739406 9801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:18.739666 9801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:18.740499 9801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:18.741714 9801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:18.741787 9801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:18.741907 9801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:18.741958 9801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:18.771958 9801 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:37205
I20260811 06:18:18.772022 10039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:37205 every 8 connection(s)
I20260811 06:18:18.773173 9801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:18:18.782179 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 9801
I20260811 06:18:18.782382 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:18:18.783183 23012 external_mini_cluster.cc:1428] Setting key f644e7c1fd88b42631a8213bc019a641
Aug 11 06:18:18 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429098, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:18:18.792517 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:18.775716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43689 (local address 127.22.121.62:35451)
0811 06:18:18.776129 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:18.776133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:18.776982 (+ 849us) server_negotiation.cc:407] Connection header received
0811 06:18:18.778730 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:18.778736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:18.778817 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:18.778912 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:18.780372 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.781097 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.783100 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.783578 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.783689 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:18.787284 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:18.787291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:18.787296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:18.787333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:18.789670 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:18.790326 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:18.790330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:18.790332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:18.790453 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:18.790854 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:18.790859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:18.790862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:18.791120 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:18.791253 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:18.791931 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 06:18:18.792184 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":210,"threads_started":1}
I20260811 06:18:18.793332 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:18.776158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35451 (local address 127.22.121.1:43689)
0811 06:18:18.776814 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:18.776857 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:18.778466 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:18.779154 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:18.779162 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:18.779658 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:18.780201 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:18.780219 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.781243 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.781247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:18.782934 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:18.782950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:18.783799 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:18.783805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:18.784055 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:18.784708 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:18.784730 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:18.787060 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:18.789801 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:18.789810 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:18.789825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:18.790212 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:18.790555 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:18.790560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:18.790564 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:18.790745 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:18.791244 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:18.791251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:18.791584 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 06:18:18.791808 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 06:18:18.792266 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":246,"threads_started":1}
I20260811 06:18:18.794785 10040 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35451
I20260811 06:18:18.795097 10040 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:18.795948 10040 heartbeater.cc:511] Master 127.22.121.62:35451 requested a full tablet report, sending...
I20260811 06:18:18.798218 9695 ts_manager.cc:194] Registered new tserver with Master: cf57a69260f64a24b9fa441c560517c9 (127.22.121.1:37205)
I20260811 06:18:18.800087 9695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:43689
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:18:18.837714 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:18:19.149057 10049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:19.149313 10049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:19.149446 10049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:19.156278 10049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:19.156371 10049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:19.156517 10049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:18:19.165536 10049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:19.166716 10049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:19.167928 10049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:19.181869 10058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:19.181960 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:19.182122 10059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:19.182811 10049 server_base.cc:1034] running on GCE node
I20260811 06:18:19.183425 10049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:19.184250 10049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:19.185432 10049 hybrid_clock.cc:648] HybridClock initialized: now 1786429099185391 us; error 32 us; skew 500 ppm
I20260811 06:18:19.185700 10049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:18:19 dist-test-slave-gjrz krb5kdc[9606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429099, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:18:19.188650 10049 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:18:19.189939 10049 webserver.cc:460] Webserver started at http://127.22.121.2:44735/ using document root <none> and password file <none>
I20260811 06:18:19.190578 10049 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:19.190693 10049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:19.190973 10049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:19.193689 10049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "578cdf774422475293fc1f8ad583685c"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "fae8e0d3977fd7d844d195df5b1e0aa7"
server_key_iv: "4506d9e3e7a5305e812091783a008d13"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.194397 10049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "578cdf774422475293fc1f8ad583685c"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "fae8e0d3977fd7d844d195df5b1e0aa7"
server_key_iv: "4506d9e3e7a5305e812091783a008d13"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.199048 10049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 06:18:19.202199 10084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.203445 10049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:18:19.203588 10049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "578cdf774422475293fc1f8ad583685c"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "fae8e0d3977fd7d844d195df5b1e0aa7"
server_key_iv: "4506d9e3e7a5305e812091783a008d13"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.203785 10049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:19.222029 10049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:19.226471 10049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:19.226729 10049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:19.227588 10049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:19.228781 10049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:19.228855 10049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.228979 10049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:19.229032 10049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.257521 10049 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:46839
I20260811 06:18:19.257575 10300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:46839 every 8 connection(s)
I20260811 06:18:19.258618 10049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:18:19.265218 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 10049
I20260811 06:18:19.265370 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:18:19.265934 23012 external_mini_cluster.cc:1428] Setting key d0c2caf9bd55fdf26efbbff57134208d
Aug 11 06:18:19 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429099, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:18:19.278095 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.260952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:38453 (local address 127.22.121.62:35451)
0811 06:18:19.261135 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.261140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.262150 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 06:18:19.263935 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.263940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.264017 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.264118 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:19.265549 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.266590 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.268590 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.269152 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.269276 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.272378 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:19.272384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:19.272388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:19.272426 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:19.275365 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.276005 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.276010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.276013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.276178 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.276536 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.276540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.276543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.276864 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:19.277000 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.277573 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.277783 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:18:19.278841 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.261304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35451 (local address 127.22.121.2:38453)
0811 06:18:19.261993 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.262029 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.263671 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.264347 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.264355 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.264822 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:19.265371 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.265394 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.266741 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.266745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.268437 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.268450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.269348 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.269354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.269553 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.270134 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:19.270160 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:19.272169 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:19.275481 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:19.275487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:19.275498 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:19.275881 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:19.276267 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:19.276270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:19.276272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:19.276431 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:19.276963 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:19.276969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:19.277248 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.277452 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.277786 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":259,"threads_started":1}
I20260811 06:18:19.280167 10305 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35451
I20260811 06:18:19.280428 10305 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:19.281091 10305 heartbeater.cc:511] Master 127.22.121.62:35451 requested a full tablet report, sending...
I20260811 06:18:19.282455 9695 ts_manager.cc:194] Registered new tserver with Master: 578cdf774422475293fc1f8ad583685c (127.22.121.2:46839)
I20260811 06:18:19.283524 9695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:38453
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:18:19.319458 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:18:19.631904 10310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:18:19.632160 10310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:18:19.632292 10310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:18:19.639199 10310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:18:19.639331 10310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:18:19.639480 10310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:18:19.648711 10310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:18:19.649848 10310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:18:19.651206 10310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:18:19.663306 10319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:19.663378 10317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:18:19.663400 10327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:18:19.664431 10310 server_base.cc:1034] running on GCE node
I20260811 06:18:19.664984 10310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:18:19.665831 10310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:18:19.667021 10310 hybrid_clock.cc:648] HybridClock initialized: now 1786429099666975 us; error 42 us; skew 500 ppm
I20260811 06:18:19.667333 10310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:18:19 dist-test-slave-gjrz krb5kdc[9606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429099, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:18:19.670323 10310 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:18:19.671615 10310 webserver.cc:460] Webserver started at http://127.22.121.3:41111/ using document root <none> and password file <none>
I20260811 06:18:19.672246 10310 fs_manager.cc:359] Metadata directory not provided
I20260811 06:18:19.672358 10310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:18:19.672627 10310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:18:19.675670 10310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "b7c309d01cd9428ba49617310f052f60"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "c7d57778207df6ce05e2f8c0729b1d08"
server_key_iv: "b2ca304bff883ed9e48c3b0a445984a6"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.676381 10310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "b7c309d01cd9428ba49617310f052f60"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "c7d57778207df6ce05e2f8c0729b1d08"
server_key_iv: "b2ca304bff883ed9e48c3b0a445984a6"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.681089 10310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 06:18:19.684181 10338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.685367 10310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:18:19.685503 10310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "b7c309d01cd9428ba49617310f052f60"
format_stamp: "Formatted at 2026-08-11 06:18:19 on dist-test-slave-gjrz"
server_key: "c7d57778207df6ce05e2f8c0729b1d08"
server_key_iv: "b2ca304bff883ed9e48c3b0a445984a6"
server_key_version: "encryptionkey@0"
I20260811 06:18:19.685674 10310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:18:19.704532 10310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:18:19.708021 10310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:18:19.708274 10310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:18:19.709070 10310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:18:19.710304 10310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:18:19.710381 10310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.710516 10310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:18:19.710568 10310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:18:19.739868 10310 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:40511
I20260811 06:18:19.739928 10532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:40511 every 8 connection(s)
I20260811 06:18:19.740998 10310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:18:19.750617 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 10310
I20260811 06:18:19.750792 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:18:19.751361 23012 external_mini_cluster.cc:1428] Setting key edff5d520a57dce42fc8d2ea58b13722
Aug 11 06:18:19 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429099, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:18:19.758672 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.743257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37447 (local address 127.22.121.62:35451)
0811 06:18:19.743511 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.743518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.744522 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 06:18:19.746211 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.746215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.746280 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.746362 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:19.747702 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.748439 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.749847 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.750254 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.750343 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.753917 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:19.753923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:19.753927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:19.753967 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:19.756327 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.756945 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.756948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.756951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.757066 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.757394 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.757397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.757399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.757648 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:19.757763 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.758286 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.758442 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:18:19.759505 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.743693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:35451 (local address 127.22.121.3:37447)
0811 06:18:19.744368 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.744409 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.745949 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.746536 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.746544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.746996 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:18:19.747569 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.747586 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.748552 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.748555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.749741 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.749752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.750465 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.750473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.750749 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.751404 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:18:19.751441 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:18:19.753721 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:18:19.756454 (+ 2733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:19.756460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:19.756470 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:19.756856 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:19.757150 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:18:19.757154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:18:19.757156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:18:19.757316 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:18:19.757733 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:18:19.757737 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:18:19.757998 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.758199 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.758519 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":255,"threads_started":1}
I20260811 06:18:19.760726 10534 heartbeater.cc:347] Connected to a master server at 127.22.121.62:35451
I20260811 06:18:19.760972 10534 heartbeater.cc:464] Registering TS with master...
I20260811 06:18:19.761605 10534 heartbeater.cc:511] Master 127.22.121.62:35451 requested a full tablet report, sending...
I20260811 06:18:19.762881 9695 ts_manager.cc:194] Registered new tserver with Master: b7c309d01cd9428ba49617310f052f60 (127.22.121.3:40511)
I20260811 06:18:19.763670 9695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:37447
I20260811 06:18:19.766295 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:18:19.782747 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.770470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.22.121.62:35451)
0811 06:18:19.770621 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.770626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.771076 (+ 450us) server_negotiation.cc:407] Connection header received
0811 06:18:19.771191 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.771194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.771247 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.771348 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:19.773115 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.773776 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.775488 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.775999 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.776093 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.778253 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:19.778258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:19.778261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:19.778289 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:19.780695 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.781230 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.781233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.781235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.781351 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.781629 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.781632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.781634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.781927 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:19.782108 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.782327 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.782479 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:18:19.786230 9695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:18:19.792827 9695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:18:19.803395 10040 heartbeater.cc:503] Master 127.22.121.62:35451 was elected leader, sending a full tablet report...
I20260811 06:18:19.815064 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.804013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:37205 (local address 127.0.0.1:50900)
0811 06:18:19.804939 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.804978 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.805155 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.806982 (+ 1827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.806990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.807034 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:18:19.807477 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.807492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.809755 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.809761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.812175 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.812192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.814137 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.814144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.814358 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.814410 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.814519 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.814696 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":1940,"threads_started":1}
I20260811 06:18:19.816690 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.804389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.22.121.1:37205)
0811 06:18:19.806388 (+ 1999us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.806395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.806422 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:18:19.806505 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.806512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.806743 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.807684 (+ 941us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:18:19.807914 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.809591 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.812390 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.813882 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.815871 (+ 1989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.816137 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.816207 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.816373 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1854,"thread_start_us":1714,"threads_started":1}
I20260811 06:18:19.821674 9921 tablet_service.cc:1467] Processing CreateTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 (DEFAULT_TABLE table=test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:18:19.822975 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a24578f4ce4cd4a5daae09336808e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:19.824677 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.804530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:40511 (local address 127.0.0.1:55124)
0811 06:18:19.805747 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.805764 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.805869 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.818360 (+ 12491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.818365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.818391 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:18:19.818773 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.818787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.820637 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.820642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.822107 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.822121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.823840 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.823847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.824016 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.824048 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.824156 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.824287 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"mutex_wait_us":1,"thread_start_us":948,"threads_started":1}
I20260811 06:18:19.824680 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.805196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.22.121.3:40511)
0811 06:18:19.817864 (+ 12668us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.817872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.817900 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:18:19.817988 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.817994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.818191 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.818361 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:18:19.818978 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.820481 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.822308 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.823636 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.823762 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.823973 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.824195 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.824351 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12508,"thread_start_us":12345,"threads_started":1}
I20260811 06:18:19.824761 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.806344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:46839 (local address 127.0.0.1:55234)
0811 06:18:19.815306 (+ 8962us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.815324 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.815419 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.817998 (+ 2579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.818003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.818031 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:18:19.818192 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.818202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.820182 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.820187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.822052 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.822067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.824044 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.824050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.824232 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.824264 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.824348 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.824497 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8852,"thread_start_us":11535,"threads_started":1}
I20260811 06:18:19.825359 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.806635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.22.121.2:46839)
0811 06:18:19.817429 (+ 10794us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.817437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.817468 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:18:19.817559 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.817566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.817782 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.818356 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:18:19.818555 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.820028 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.822215 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.823636 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.823762 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.823932 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.824961 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.825131 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10665,"thread_start_us":10570,"threads_started":1}
I20260811 06:18:19.828217 10180 tablet_service.cc:1467] Processing CreateTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 (DEFAULT_TABLE table=test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:18:19.828469 10411 tablet_service.cc:1467] Processing CreateTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 (DEFAULT_TABLE table=test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:18:19.829252 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a24578f4ce4cd4a5daae09336808e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:19.829721 10411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a24578f4ce4cd4a5daae09336808e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:18:19.837599 10567 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Bootstrap starting.
I20260811 06:18:19.839761 10571 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Bootstrap starting.
I20260811 06:18:19.841200 10567 tablet_bootstrap.cc:654] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:19.842640 10567 log.cc:824] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:18:19.843101 10571 tablet_bootstrap.cc:654] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:19.844373 10571 log.cc:824] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:18:19.845095 10572 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Bootstrap starting.
I20260811 06:18:19.845489 10567 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: No bootstrap required, opened a new log
I20260811 06:18:19.845746 10567 ts_tablet_manager.cc:1397] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 06:18:19.847187 10571 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: No bootstrap required, opened a new log
I20260811 06:18:19.847472 10571 ts_tablet_manager.cc:1397] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:18:19.848248 10572 tablet_bootstrap.cc:654] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Neither blocks nor log segments found. Creating new log.
I20260811 06:18:19.849380 10572 log.cc:824] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Log is configured to *not* fsync() on all Append() calls
I20260811 06:18:19.852129 10572 tablet_bootstrap.cc:492] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: No bootstrap required, opened a new log
I20260811 06:18:19.852422 10572 ts_tablet_manager.cc:1397] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260811 06:18:19.852999 10567 raft_consensus.cc:348] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.853315 10567 raft_consensus.cc:374] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:19.853415 10567 raft_consensus.cc:729] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf57a69260f64a24b9fa441c560517c9, State: Initialized, Role: FOLLOWER
I20260811 06:18:19.853955 10567 consensus_queue.cc:260] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.855585 10567 ts_tablet_manager.cc:1428] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 06:18:19.856801 10571 raft_consensus.cc:348] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.857187 10571 raft_consensus.cc:374] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:19.857285 10571 raft_consensus.cc:729] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578cdf774422475293fc1f8ad583685c, State: Initialized, Role: FOLLOWER
I20260811 06:18:19.858587 10571 consensus_queue.cc:260] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.859658 10305 heartbeater.cc:503] Master 127.22.121.62:35451 was elected leader, sending a full tablet report...
I20260811 06:18:19.860158 10571 ts_tablet_manager.cc:1428] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 06:18:19.862249 10572 raft_consensus.cc:348] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.862612 10572 raft_consensus.cc:374] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:18:19.862701 10572 raft_consensus.cc:729] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c309d01cd9428ba49617310f052f60, State: Initialized, Role: FOLLOWER
I20260811 06:18:19.863185 10572 consensus_queue.cc:260] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.864065 10534 heartbeater.cc:503] Master 127.22.121.62:35451 was elected leader, sending a full tablet report...
I20260811 06:18:19.864423 10572 ts_tablet_manager.cc:1428] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Time spent starting tablet: real 0.012s user 0.008s sys 0.002s
I20260811 06:18:19.881136 10582 raft_consensus.cc:482] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:18:19.881330 10582 raft_consensus.cc:504] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.882745 10582 leader_election.cc:288] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 578cdf774422475293fc1f8ad583685c (127.22.121.2:46839), b7c309d01cd9428ba49617310f052f60 (127.22.121.3:40511)
I20260811 06:18:19.889168 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.883083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:47657 (local address 127.22.121.3:40511)
0811 06:18:19.883307 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.883313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.883956 (+ 643us) server_negotiation.cc:407] Connection header received
0811 06:18:19.884096 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.884101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.884167 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.884245 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:18:19.884856 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.885634 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.887485 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.888262 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.888380 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.888430 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.888806 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.888931 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:18:19.889245 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.883350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:56481 (local address 127.22.121.2:46839)
0811 06:18:19.883527 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.883532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.884105 (+ 573us) server_negotiation.cc:407] Connection header received
0811 06:18:19.884262 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.884266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.884328 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.884412 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:18:19.884967 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.885648 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.887565 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.888379 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.888518 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.888587 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.888889 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.889009 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:18:19.889322 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.883026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:40511 (local address 127.22.121.1:47657)
0811 06:18:19.883849 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.883875 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.883982 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.884264 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.884269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.884293 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:18:19.884713 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.884726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.885760 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.885765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.887345 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.887362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.888453 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.888459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.888655 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.888679 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.888766 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.888888 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":109,"thread_start_us":446,"threads_started":1}
I20260811 06:18:19.889554 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.883122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:46839 (local address 127.22.121.1:56481)
0811 06:18:19.884002 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 06:18:19.884016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:18:19.884147 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:18:19.884452 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:18:19.884455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:18:19.884479 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:18:19.884818 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.884830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.885773 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.885777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.887424 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:18:19.887440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.888583 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.888608 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:18:19.888740 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.888771 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:18:19.888858 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:18:19.888968 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":18,"thread_start_us":565,"threads_started":1}
I20260811 06:18:19.890194 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a24578f4ce4cd4a5daae09336808e2" candidate_uuid: "cf57a69260f64a24b9fa441c560517c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c309d01cd9428ba49617310f052f60" is_pre_election: true
I20260811 06:18:19.890471 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a24578f4ce4cd4a5daae09336808e2" candidate_uuid: "cf57a69260f64a24b9fa441c560517c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578cdf774422475293fc1f8ad583685c" is_pre_election: true
I20260811 06:18:19.890702 10450 raft_consensus.cc:2454] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf57a69260f64a24b9fa441c560517c9 in term 0.
I20260811 06:18:19.890942 10215 raft_consensus.cc:2454] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf57a69260f64a24b9fa441c560517c9 in term 0.
I20260811 06:18:19.891378 9830 leader_election.cc:302] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7c309d01cd9428ba49617310f052f60, cf57a69260f64a24b9fa441c560517c9; no voters:
I20260811 06:18:19.891717 10582 raft_consensus.cc:2781] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:18:19.891839 10582 raft_consensus.cc:482] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:18:19.891938 10582 raft_consensus.cc:3037] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:19.893983 10582 raft_consensus.cc:504] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.894532 10582 leader_election.cc:288] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [CANDIDATE]: Term 1 election: Requested vote from peers 578cdf774422475293fc1f8ad583685c (127.22.121.2:46839), b7c309d01cd9428ba49617310f052f60 (127.22.121.3:40511)
I20260811 06:18:19.894865 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a24578f4ce4cd4a5daae09336808e2" candidate_uuid: "cf57a69260f64a24b9fa441c560517c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578cdf774422475293fc1f8ad583685c"
I20260811 06:18:19.894990 10450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a24578f4ce4cd4a5daae09336808e2" candidate_uuid: "cf57a69260f64a24b9fa441c560517c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7c309d01cd9428ba49617310f052f60"
I20260811 06:18:19.895056 10215 raft_consensus.cc:3037] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:19.895171 10450 raft_consensus.cc:3037] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:18:19.898186 10450 raft_consensus.cc:2454] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf57a69260f64a24b9fa441c560517c9 in term 1.
I20260811 06:18:19.898186 10215 raft_consensus.cc:2454] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf57a69260f64a24b9fa441c560517c9 in term 1.
I20260811 06:18:19.898583 9837 leader_election.cc:302] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 578cdf774422475293fc1f8ad583685c, b7c309d01cd9428ba49617310f052f60, cf57a69260f64a24b9fa441c560517c9; no voters:
I20260811 06:18:19.898866 10582 raft_consensus.cc:2781] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:18:19.899322 10582 raft_consensus.cc:686] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 1 LEADER]: Becoming Leader. State: Replica: cf57a69260f64a24b9fa441c560517c9, State: Running, Role: LEADER
I20260811 06:18:19.900012 10582 consensus_queue.cc:237] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } }
I20260811 06:18:19.904137 9695 catalog_manager.cc:5495] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf57a69260f64a24b9fa441c560517c9 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "cf57a69260f64a24b9fa441c560517c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf57a69260f64a24b9fa441c560517c9" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 } health_report { overall_health: UNKNOWN } } }
I20260811 06:18:19.933914 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.929329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.22.121.1:37205)
0811 06:18:19.929531 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.929538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.929645 (+ 107us) server_negotiation.cc:407] Connection header received
0811 06:18:19.929815 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.929819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.929889 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.929997 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:18:19.930275 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.930945 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.932184 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.932581 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.932670 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.933041 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:18:19.933464 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:18:19.933583 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.933624 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.933681 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:18:19.941165 10450 raft_consensus.cc:1261] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 1 FOLLOWER]: Refusing update from remote peer cf57a69260f64a24b9fa441c560517c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:18:19.941166 10215 raft_consensus.cc:1261] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 1 FOLLOWER]: Refusing update from remote peer cf57a69260f64a24b9fa441c560517c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:18:19.942090 10589 consensus_queue.cc:1035] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7c309d01cd9428ba49617310f052f60" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 40511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:18:19.942392 10582 consensus_queue.cc:1035] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578cdf774422475293fc1f8ad583685c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:18:19.954507 10592 mvcc.cc:205] Tried to move back new op lower bound from 7317213593350332416 to 7317213593193611264. Current Snapshot: MvccSnapshot[applied={T|T < 7317213593350332416}]
I20260811 06:18:19.955143 10595 mvcc.cc:205] Tried to move back new op lower bound from 7317213593350332416 to 7317213593193611264. Current Snapshot: MvccSnapshot[applied={T|T < 7317213593350332416}]
I20260811 06:18:19.966181 9695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:18:19.966383 9695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:18:19.969717 9695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8091ccf5cb684c36a15dfafc96cf082f: Sending DeleteTablet for 3 replicas of tablet 39a24578f4ce4cd4a5daae09336808e2
I20260811 06:18:19.970809 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:18:19 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:55234
I20260811 06:18:19.970813 9921 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:18:19 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:50900
I20260811 06:18:19.971481 10411 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a24578f4ce4cd4a5daae09336808e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:18:19 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:55124
I20260811 06:18:19.972373 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 9801
I20260811 06:18:19.974205 10622 tablet_replica.cc:330] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: stopping tablet replica
I20260811 06:18:19.974643 10622 raft_consensus.cc:2229] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:18:19.975163 10622 raft_consensus.cc:2258] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:18:19.975988 10615 tablet_replica.cc:330] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: stopping tablet replica
I20260811 06:18:19.976389 10615 raft_consensus.cc:2229] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:18:19.976869 10615 raft_consensus.cc:2258] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:18:19.977600 10620 tablet_replica.cc:330] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: stopping tablet replica
I20260811 06:18:19.977980 10620 raft_consensus.cc:2229] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:18:19.978542 10620 raft_consensus.cc:2258] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:18:19.978902 10615 ts_tablet_manager.cc:1905] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:18:19.980729 10620 ts_tablet_manager.cc:1905] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:18:19.986048 10615 ts_tablet_manager.cc:1918] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:18:19.986212 10615 log.cc:1196] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/39a24578f4ce4cd4a5daae09336808e2
I20260811 06:18:19.986711 10615 ts_tablet_manager.cc:1939] T 39a24578f4ce4cd4a5daae09336808e2 P 578cdf774422475293fc1f8ad583685c: Deleting consensus metadata
I20260811 06:18:19.988229 10620 ts_tablet_manager.cc:1918] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:18:19.988452 10620 log.cc:1196] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/39a24578f4ce4cd4a5daae09336808e2
I20260811 06:18:19.988395 9670 catalog_manager.cc:4841] TS 578cdf774422475293fc1f8ad583685c (127.22.121.2:46839): tablet 39a24578f4ce4cd4a5daae09336808e2 (table test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]) successfully deleted
I20260811 06:18:19.989051 10620 ts_tablet_manager.cc:1939] T 39a24578f4ce4cd4a5daae09336808e2 P cf57a69260f64a24b9fa441c560517c9: Deleting consensus metadata
I20260811 06:18:19.989394 10622 ts_tablet_manager.cc:1905] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:18:19.991031 9670 catalog_manager.cc:4841] TS cf57a69260f64a24b9fa441c560517c9 (127.22.121.1:37205): tablet 39a24578f4ce4cd4a5daae09336808e2 (table test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]) successfully deleted
Aug 11 06:18:19 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:18:19.996006 10622 ts_tablet_manager.cc:1918] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:18:19.996166 10622 log.cc:1196] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/39a24578f4ce4cd4a5daae09336808e2
I20260811 06:18:19.996671 10622 ts_tablet_manager.cc:1939] T 39a24578f4ce4cd4a5daae09336808e2 P b7c309d01cd9428ba49617310f052f60: Deleting consensus metadata
I20260811 06:18:19.998245 9666 catalog_manager.cc:4841] TS b7c309d01cd9428ba49617310f052f60 (127.22.121.3:40511): tablet 39a24578f4ce4cd4a5daae09336808e2 (table test-table [id=c5bfb0ce51af492ea3dea8bec8ca5255]) successfully deleted
I20260811 06:18:19.999742 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:19.973517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.22.121.1:37205)
0811 06:18:19.973748 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:19.973754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:19.983413 (+ 9659us) server_negotiation.cc:407] Connection header received
0811 06:18:19.983829 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:19.983835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:19.983906 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:19.983967 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:19.985649 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.986907 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.988775 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:19.989489 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:19.989596 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:19.994138 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:19.994149 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:19.994161 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:19.994190 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:19.997075 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.997686 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.997692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.997695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.997826 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:19.998250 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:19.998254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:19.998256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:19.998625 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:19.998904 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:19.999191 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 06:18:19.999437 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 06:18:20.000340 10029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:20.093354 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 9801
I20260811 06:18:20.114918 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 10049
Aug 11 06:18:20 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:18:20.128604 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:20.115789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.22.121.2:46839)
0811 06:18:20.115975 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:20.115979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:20.115998 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:18:20.116065 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:20.116069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:20.116141 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:20.116220 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:20.117405 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:20.118171 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:20.119634 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:20.120026 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:20.120125 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:20.123848 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:20.123861 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:20.123877 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:20.123910 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:20.126363 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:20.126921 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:20.126927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:20.126930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:20.127047 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:20.127354 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:20.127357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:20.127359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:20.127706 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:20.127915 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:20.128151 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 06:18:20.128305 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:18:20.129256 10293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:20.230065 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 10049
I20260811 06:18:20.249866 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 10310
Aug 11 06:18:20 dist-test-slave-gjrz krb5kdc[9606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:18:20.264178 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:18:20.250672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.22.121.3:40511)
0811 06:18:20.250836 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:18:20.250841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:18:20.250858 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:18:20.250915 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:18:20.250918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:18:20.250979 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:18:20.251070 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:18:20.252289 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:20.253001 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:20.254152 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:18:20.254528 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:18:20.254613 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:18:20.258498 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:18:20.258514 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:18:20.258531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:18:20.258562 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:18:20.261566 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:20.262196 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:20.262206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:20.262210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:20.262342 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:18:20.262687 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:18:20.262693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:18:20.262697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:18:20.263046 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:18:20.263396 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:18:20.263660 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:18:20.263871 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:18:20.264765 10526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:20.351521 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 10310
I20260811 06:18:20.375010 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 9630
I20260811 06:18:20.376143 9742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:18:20.481307 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 9630
2026-08-11T06:18:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4190 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:18:20 dist-test-slave-gjrz krb5kdc[10640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:18:20 dist-test-slave-gjrz krb5kdc[10640](info): set up 2 sockets
Aug 11 06:18:20 dist-test-slave-gjrz krb5kdc[10640](info): commencing operation
krb5kdc: starting...
W20260811 06:18:21.726164 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:18:21 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:18:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangeradmin_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangeradmin_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangerlookup_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangerlookup_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
I20260811 06:18:22.018046 23012 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTjGG_n/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/postgres -l logfile start
2026-08-11 06:18:24.308 UTC [10693] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:18:24.308 UTC [10693] LOG: listening on IPv4 address "127.22.121.20", port 53967
2026-08-11 06:18:24.311 UTC [10693] LOG: listening on Unix socket "/tmp/.s.PGSQL.53967"
2026-08-11 06:18:24.332 UTC [10697] LOG: database system was shut down at 2026-08-11 06:18:23 UTC
2026-08-11 06:18:24.336 UTC [10693] LOG: database system is ready to accept connections
I20260811 06:18:25.447297 23012 mini_postgres.cc:96] Postgres bound to 53967
2026-08-11 06:18:25.469 UTC [10717] FATAL: database "slave" does not exist
127.22.121.20:53967 - accepting connections
I20260811 06:18:25.470130 23012 mini_ranger.cc:161] Starting Ranger...
I20260811 06:18:25.497109 23012 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:18:25.910369 23012 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:18:25.910511 23012 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin
2026-08-11 06:18:26,281 [I] DB FLAVOR :POSTGRES
2026-08-11 06:18:26,281 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:18:26,282 [I] Checking connection..
2026-08-11 06:18:26,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:18:26,530 [I] Checking connection passed.
2026-08-11 06:18:26,531 [I] --------- Verifying version history table ---------
2026-08-11 06:18:26,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:18:26,782 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:18:26,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:18:27,024 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:18:27,025 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:18:27,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:18:27.270 UTC [10851] WARNING: there is no transaction in progress
2026-08-11 06:18:27,278 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:18:27,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:18:27,529 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:18:27,529 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:18:27,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:18:27,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:18:28,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-11 06:18:28,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:18:28,470 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:18:28,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:18:28,718 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:18:28,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:18:28,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:18:29,199 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:18:29,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:18:30.060 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.075 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.089 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.102 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.116 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.133 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.185 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.191 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.203 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.212 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.221 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.233 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.242 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.248 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.254 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30.260 UTC [11041] WARNING: there is no transaction in progress
2026-08-11 06:18:30,801 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:18:30,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:18:31,041 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:18:31,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:18:31,290 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:18:31,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:18:31,530 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:18:31,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:18:31,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:18:32,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-11 06:18:32,250 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:18:32,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:18:32,481 [I] DB_PATCHES have already been applied
I20260811 06:18:32.487155 23012 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTjGG_n/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:18:32.487287 23012 mini_ranger.cc:193] Using host: 127.22.121.20
I20260811 06:18:32.505661 23012 mini_ranger.cc:227] Ranger admin URL: http://127.22.121.20:33331
Aug 11, 2026 6:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:18:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:18:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:18:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:18:33.820618 23012 mini_ranger.cc:160] Time spent starting Ranger: real 8.350s user 0.009s sys 0.040s
Aug 11, 2026 6:18:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:18:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:18:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:18:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:18:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:18:54.493781 23012 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangerkms_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/rangerkms_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.121.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.121.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:18:54 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429134, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:18:54.652238 23012 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:18:54.670208 23012 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:18:55.426601 23012 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:18:55.426730 23012 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms
I20260811 06:18:55.426863 23012 mini_ranger_kms.cc:227] Using postgres at 127.22.121.20:53967
2026-08-11 06:18:55,907 [I] DB FLAVOR :POSTGRES
2026-08-11 06:18:55,907 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:18:55,907 [I] Checking connection
2026-08-11 06:18:55,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:18:56,152 [I] connection success
2026-08-11 06:18:56,153 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:18:56,153 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:18:56,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:18:56,413 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:18:56,414 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:18:56,414 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:18:56,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:53967/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:18:56,747 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:18:57.167977 23012 mini_ranger_kms.cc:310] Created kms service
I20260811 06:18:57.362133 23012 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:18:57.404593 23012 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:18:57 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429134, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:18:57.774971 23012 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:18:57.776888 23012 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:18:57.776971 23012 mini_ranger_kms.cc:240] Using host: 127.22.121.20
I20260811 06:18:57.799981 23012 mini_ranger_kms.cc:276] Ranger KMS PID: 11350
I20260811 06:18:57.800155 23012 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.121.20:45765
Aug 11, 2026 6:18:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:18:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:18:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:18:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:18:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:18:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:19:00 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:00.387854 23012 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.736s user 0.115s sys 0.390s
I20260811 06:19:00.388254 23012 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:19:00.388481 23012 mini_ranger_kms.cc:401] 127.22.121.20:45765/kms/v1/keys
Aug 11 06:19:00 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
Aug 11 06:19:00 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429140, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
Aug 11 06:19:03 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:19:03.304875 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45699
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:45699
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:45765/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:19:03.715415 11433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:03.715729 11433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:03.715837 11433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:03.723646 11433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:19:03.723767 11433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:03.723831 11433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:03.723891 11433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:19:03.723948 11433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:19:03.734570 11433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.121.20:45765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:45699
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:03.735854 11433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:03.737077 11433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:03.749531 11440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:03.749580 11446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:03.749607 11441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:03.750460 11433 server_base.cc:1034] running on GCE node
I20260811 06:19:03.751303 11433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:03.752650 11433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:03.753880 11433 hybrid_clock.cc:648] HybridClock initialized: now 1786429143753826 us; error 36 us; skew 500 ppm
I20260811 06:19:03.754228 11433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:03 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:03.757375 11433 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:19:03.758924 11433 webserver.cc:460] Webserver started at http://127.22.121.62:36655/ using document root <none> and password file <none>
I20260811 06:19:03.759543 11433 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:03.759692 11433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:03.760007 11433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:03 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:03.824357 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "569ace5ff60347159da6ab890db3c144"
format_stamp: "Formatted at 2026-08-11 06:19:03 on dist-test-slave-gjrz"
server_key: "bf6c48e3caa036d7e7d683e3d024593a"
server_key_iv: "99ecc529501bf781207f2cb3fbfc8be9"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:03.825374 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "569ace5ff60347159da6ab890db3c144"
format_stamp: "Formatted at 2026-08-11 06:19:03 on dist-test-slave-gjrz"
server_key: "bf6c48e3caa036d7e7d683e3d024593a"
server_key_iv: "99ecc529501bf781207f2cb3fbfc8be9"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:03.831955 11433 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 06:19:03.852988 11460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:03.854454 11433 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:19:03.854641 11433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "569ace5ff60347159da6ab890db3c144"
format_stamp: "Formatted at 2026-08-11 06:19:03 on dist-test-slave-gjrz"
server_key: "bf6c48e3caa036d7e7d683e3d024593a"
server_key_iv: "99ecc529501bf781207f2cb3fbfc8be9"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:03.854884 11433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:03.872071 11433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:03.876549 11433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:03.876948 11433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:03.906739 11433 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:45699
I20260811 06:19:03.906811 11551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:45699 every 8 connection(s)
I20260811 06:19:03.908082 11433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:19:03.912983 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 11433
I20260811 06:19:03.913137 11554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:03.913221 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:19:03.914050 23012 external_mini_cluster.cc:1428] Setting key 954662c9e08a1cfdcdfca9c9fa0e7310
I20260811 06:19:03.928306 11554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: Bootstrap starting.
Aug 11 06:19:03 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:03.933176 11554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:03.934732 11554 log.cc:824] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:03.938230 11554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: No bootstrap required, opened a new log
I20260811 06:19:03.940120 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:03.916040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.22.121.62:45699)
0811 06:19:03.917403 (+ 1363us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:03.917415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:03.917455 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:19:03.920492 (+ 3037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:03.920517 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:03.921098 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:03.921424 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:03.922928 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:03.924403 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:03.926125 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:03.926958 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:03.927093 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:03.931350 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:03.931361 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:03.931375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:03.931413 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:03.935696 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:03.936333 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:03.936344 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:03.936350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:03.936590 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:03.936957 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:03.936961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:03.936963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:03.937527 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:03.937954 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:03.938407 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:19:03.938755 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1088,"thread_start_us":723,"threads_started":1}
I20260811 06:19:03.948093 11554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } }
I20260811 06:19:03.948628 11554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:03.948830 11554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 569ace5ff60347159da6ab890db3c144, State: Initialized, Role: FOLLOWER
I20260811 06:19:03.949604 11554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } }
I20260811 06:19:03.949950 11554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:19:03.950160 11554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:19:03.950395 11554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:03.953272 11554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } }
I20260811 06:19:03.953760 11554 leader_election.cc:302] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 569ace5ff60347159da6ab890db3c144; no voters:
I20260811 06:19:03.954387 11554 leader_election.cc:288] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:19:03.954767 11561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:19:03.956318 11561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [term 1 LEADER]: Becoming Leader. State: Replica: 569ace5ff60347159da6ab890db3c144, State: Running, Role: LEADER
I20260811 06:19:03.957295 11554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:19:03.957183 11561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } }
I20260811 06:19:03.960763 11563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 569ace5ff60347159da6ab890db3c144. Latest consensus state: current_term: 1 leader_uuid: "569ace5ff60347159da6ab890db3c144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } } }
I20260811 06:19:03.961251 11563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:03.960733 11562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "569ace5ff60347159da6ab890db3c144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "569ace5ff60347159da6ab890db3c144" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45699 } } }
I20260811 06:19:03.961546 11562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:03.963922 11568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:19:03.970333 11568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:19:03.971905 11554 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:19:03.990309 11568 catalog_manager.cc:1347] Generated new cluster ID: f7d083e1d9024da29894e3c31d378cd6
I20260811 06:19:03.990511 11568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:19:04.015810 11568 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:19:04.017976 11568 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:19:04.032168 11568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: Generated new TSK 0
I20260811 06:19:04.033279 11568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:19:04.786154 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:19:05.249944 11630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:05.250353 11630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:05.250516 11630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:05.262326 11630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:05.262466 11630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:05.262666 11630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:19:05.278118 11630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.121.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:05.279781 11630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:05.281741 11630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:05.309506 11647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:05.309734 11649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:05.317729 11655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:05.318132 11630 server_base.cc:1034] running on GCE node
I20260811 06:19:05.318964 11630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:05.320307 11630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:05.321673 11630 hybrid_clock.cc:648] HybridClock initialized: now 1786429145321521 us; error 96 us; skew 500 ppm
I20260811 06:19:05.322098 11630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:05.326422 11630 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:19:05.328433 11630 webserver.cc:460] Webserver started at http://127.22.121.1:37375/ using document root <none> and password file <none>
I20260811 06:19:05.329424 11630 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:05.329571 11630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:05.329936 11630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:05.351594 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "86013ca7b13f4c8781a908d24ebe445d"
format_stamp: "Formatted at 2026-08-11 06:19:05 on dist-test-slave-gjrz"
server_key: "8155a92306475cd69155d3285c5e367b"
server_key_iv: "7e82893e937961d9e309728f27209f25"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:05.354202 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "86013ca7b13f4c8781a908d24ebe445d"
format_stamp: "Formatted at 2026-08-11 06:19:05 on dist-test-slave-gjrz"
server_key: "8155a92306475cd69155d3285c5e367b"
server_key_iv: "7e82893e937961d9e309728f27209f25"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:05.366791 11630 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.011s sys 0.000s
I20260811 06:19:05.388460 11663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:05.390376 11630 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.004s
I20260811 06:19:05.390571 11630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "86013ca7b13f4c8781a908d24ebe445d"
format_stamp: "Formatted at 2026-08-11 06:19:05 on dist-test-slave-gjrz"
server_key: "8155a92306475cd69155d3285c5e367b"
server_key_iv: "7e82893e937961d9e309728f27209f25"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:05.390833 11630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:05.427417 11630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:05.433908 11630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:05.434507 11630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:05.437234 11630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:05.440447 11630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:05.440605 11630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:05.440793 11630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:05.440886 11630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:05.647100 11861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33265 every 8 connection(s)
I20260811 06:19:05.647557 11630 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33265
I20260811 06:19:05.649449 11630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:19:05.652792 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 11630
I20260811 06:19:05.652998 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:19:05.653842 23012 external_mini_cluster.cc:1428] Setting key ab7f83092c6d76fcbb7ff90276741c51
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:19:05.713477 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:05.659571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:55613 (local address 127.22.121.62:45699)
0811 06:19:05.677671 (+ 18100us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:05.677677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:05.686713 (+ 9036us) server_negotiation.cc:407] Connection header received
0811 06:19:05.686784 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:05.686789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:05.686886 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:05.686996 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:05.692436 (+ 5440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:05.693626 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:05.696854 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:05.697559 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:05.697661 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:05.706188 (+ 8527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:05.706196 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:05.706201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:05.706239 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:05.709887 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:05.710876 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:05.710882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:05.710885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:05.711037 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:05.711508 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:05.711513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:05.711516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:05.711924 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:05.712070 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:05.712743 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 06:19:05.713062 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17953,"thread_start_us":18102,"threads_started":1}
I20260811 06:19:05.714404 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:05.658910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45699 (local address 127.22.121.1:55613)
0811 06:19:05.683706 (+ 24796us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:05.683763 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:05.686324 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:05.688126 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:05.688138 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:05.688857 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:05.689688 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:05.689714 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:05.693850 (+ 4136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:05.693855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:05.696202 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:05.696219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:05.698070 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:05.698077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:05.698345 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:05.703457 (+ 5112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:05.703495 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:05.705948 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:05.710067 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:05.710077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:05.710093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:05.710720 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:05.711154 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:05.711158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:05.711160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:05.711393 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:05.712054 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:05.712062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:05.712382 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 06:19:05.712622 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 06:19:05.713103 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24481,"thread_start_us":18905,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
I20260811 06:19:05.716243 11862 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45699
I20260811 06:19:05.716545 11862 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:05.717348 11862 heartbeater.cc:511] Master 127.22.121.62:45699 requested a full tablet report, sending...
I20260811 06:19:05.719836 11489 ts_manager.cc:194] Registered new tserver with Master: 86013ca7b13f4c8781a908d24ebe445d (127.22.121.1:33265)
I20260811 06:19:05.721788 11489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:55613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:19:05.746073 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:05 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429145, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
W20260811 06:19:06.258343 11876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:06.258827 11876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:06.259076 11876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:06.270831 11876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:06.271041 11876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:06.271248 11876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:19:06.281563 11876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.121.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:06.282850 11876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:06.284346 11876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:06.302029 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:06.302078 11901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:06.303691 11899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:06.304982 11876 server_base.cc:1034] running on GCE node
I20260811 06:19:06.305763 11876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:06.306746 11876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:06.307974 11876 hybrid_clock.cc:648] HybridClock initialized: now 1786429146307912 us; error 49 us; skew 500 ppm
I20260811 06:19:06.308288 11876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:06 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429146, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:06.312242 11876 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:19:06.313900 11876 webserver.cc:460] Webserver started at http://127.22.121.2:40861/ using document root <none> and password file <none>
I20260811 06:19:06.314554 11876 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:06.314728 11876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:06.315164 11876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:06 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429146, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:06.334266 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "b4e6af1b230d446e8e72e8716f37bb87"
format_stamp: "Formatted at 2026-08-11 06:19:06 on dist-test-slave-gjrz"
server_key: "a7e5de87cfb56cb5e6688b180a4b349e"
server_key_iv: "65eeedea7852e38d068009a4f8095140"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:06.335712 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "b4e6af1b230d446e8e72e8716f37bb87"
format_stamp: "Formatted at 2026-08-11 06:19:06 on dist-test-slave-gjrz"
server_key: "a7e5de87cfb56cb5e6688b180a4b349e"
server_key_iv: "65eeedea7852e38d068009a4f8095140"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:06.343827 11876 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 06:19:06.363720 11919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:06.365501 11876 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 06:19:06.365733 11876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "b4e6af1b230d446e8e72e8716f37bb87"
format_stamp: "Formatted at 2026-08-11 06:19:06 on dist-test-slave-gjrz"
server_key: "a7e5de87cfb56cb5e6688b180a4b349e"
server_key_iv: "65eeedea7852e38d068009a4f8095140"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:06.366057 11876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:06.404547 11876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:06.411522 11876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:06.412031 11876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:06.414041 11876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:06.416379 11876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:06.416602 11876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:19:06.416819 11876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:06.416963 11876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:06.495635 11876 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:38525
I20260811 06:19:06.495653 12100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:38525 every 8 connection(s)
I20260811 06:19:06.497186 11876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:19:06.503655 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 11876
I20260811 06:19:06.503832 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:19:06.504341 23012 external_mini_cluster.cc:1428] Setting key 8dcff4ade59f469fcc42a13220611eb4
Aug 11 06:19:06 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429146, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:06.523989 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:06.500039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:41507 (local address 127.22.121.62:45699)
0811 06:19:06.500524 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:06.500529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:06.501723 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 06:19:06.503791 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:06.503797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:06.503888 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:06.504005 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:06.505730 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:06.506608 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:06.509458 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:06.510002 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:06.510124 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:06.515088 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:06.515096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:06.515101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:06.515141 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:06.520052 (+ 4911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:06.520892 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:06.520898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:06.520902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:06.521118 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:06.521723 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:06.521729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:06.521733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:06.522218 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:06.522370 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:06.523334 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 06:19:06.523604 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":252,"threads_started":1}
I20260811 06:19:06.525086 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:06.500759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45699 (local address 127.22.121.2:41507)
0811 06:19:06.501567 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:06.501604 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:06.503455 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:06.504266 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:06.504278 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:06.504868 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:06.505572 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:06.505590 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:06.506782 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:06.506792 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:06.509279 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:06.509295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:06.510323 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:06.510332 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:06.510749 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:06.511636 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:06.511667 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:06.514836 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:06.520209 (+ 5373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:06.520218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:06.520233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:06.520756 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:06.521247 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:06.521252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:06.521255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:06.521596 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:06.522369 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:06.522378 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:06.522831 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 06:19:06.523121 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 06:19:06.523726 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":352,"threads_started":1}
I20260811 06:19:06.526585 12102 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45699
I20260811 06:19:06.526902 12102 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:06.527809 12102 heartbeater.cc:511] Master 127.22.121.62:45699 requested a full tablet report, sending...
I20260811 06:19:06.529538 11489 ts_manager.cc:194] Registered new tserver with Master: b4e6af1b230d446e8e72e8716f37bb87 (127.22.121.2:38525)
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
I20260811 06:19:06.530697 11489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:41507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:19:06.559371 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:19:06.725448 11862 heartbeater.cc:503] Master 127.22.121.62:45699 was elected leader, sending a full tablet report...
W20260811 06:19:07.030406 12112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:07.030738 12112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:07.030926 12112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:07.040791 12112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:07.040967 12112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:07.041265 12112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:19:07.054181 12112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.121.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:07.055502 12112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:07.056949 12112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:07.073527 12123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:07.074116 12124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:07.074782 12127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:07.075578 12112 server_base.cc:1034] running on GCE node
I20260811 06:19:07.076242 12112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:07.077306 12112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:07.078541 12112 hybrid_clock.cc:648] HybridClock initialized: now 1786429147078458 us; error 29 us; skew 500 ppm
I20260811 06:19:07.078940 12112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:07 dist-test-slave-gjrz krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429147, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:07.083209 12112 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:19:07.084966 12112 webserver.cc:460] Webserver started at http://127.22.121.3:37291/ using document root <none> and password file <none>
I20260811 06:19:07.085747 12112 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:07.085948 12112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:07.086350 12112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:07 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429147, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:07.103082 12112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "cb5723272d51495fb045212da92fe130"
format_stamp: "Formatted at 2026-08-11 06:19:07 on dist-test-slave-gjrz"
server_key: "a8b56dfc5f1fc702c31e2ed0565a9c07"
server_key_iv: "4c8f6fa15404731193944ccb59b6b240"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:07.103994 12112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "cb5723272d51495fb045212da92fe130"
format_stamp: "Formatted at 2026-08-11 06:19:07 on dist-test-slave-gjrz"
server_key: "a8b56dfc5f1fc702c31e2ed0565a9c07"
server_key_iv: "4c8f6fa15404731193944ccb59b6b240"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:07.110085 12112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 06:19:07.123888 12137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:07.125146 12112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:19:07.125360 12112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "cb5723272d51495fb045212da92fe130"
format_stamp: "Formatted at 2026-08-11 06:19:07 on dist-test-slave-gjrz"
server_key: "a8b56dfc5f1fc702c31e2ed0565a9c07"
server_key_iv: "4c8f6fa15404731193944ccb59b6b240"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:07.125607 12112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:07.149200 12112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:07.152881 12112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:07.153291 12112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:07.154384 12112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:07.155833 12112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:07.155915 12112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:07.156031 12112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:07.156086 12112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:07.203457 12112 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:35683
I20260811 06:19:07.203564 12327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:35683 every 8 connection(s)
I20260811 06:19:07.205474 12112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:19:07.209843 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 12112
I20260811 06:19:07.210008 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:19:07.210747 23012 external_mini_cluster.cc:1428] Setting key 829f47d67535ed28e93404fa7c70b62d
Aug 11 06:19:07 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429147, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:07.238840 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.209112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:55501 (local address 127.22.121.62:45699)
0811 06:19:07.212665 (+ 3553us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.212671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.216381 (+ 3710us) server_negotiation.cc:407] Connection header received
0811 06:19:07.219185 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.219191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.219332 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.219429 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:07.221255 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.222136 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.224340 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.225337 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.225462 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.230438 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:07.230446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:07.230452 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:07.230503 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:07.235164 (+ 4661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.236007 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.236013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.236017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.236189 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.236703 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.236708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.236712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.237056 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:07.237289 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.237984 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.238253 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3407,"thread_start_us":3703,"threads_started":1}
I20260811 06:19:07.240149 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.209914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45699 (local address 127.22.121.3:55501)
0811 06:19:07.216222 (+ 6308us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.216268 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.218885 (+ 2617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.219727 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.219736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.220354 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:07.221089 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.221109 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.222333 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.222336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.224140 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.224184 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.225653 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.225663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.226031 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.227196 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:07.227240 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:07.230121 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:07.235322 (+ 5201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:07.235329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:07.235340 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:07.235870 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:07.236288 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:07.236291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:07.236293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:07.236598 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:07.237170 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:07.237175 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:07.237575 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.237911 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.238498 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6013,"thread_start_us":5643,"threads_started":1}
I20260811 06:19:07.241477 12330 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45699
I20260811 06:19:07.241797 12330 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:07.242795 12330 heartbeater.cc:511] Master 127.22.121.62:45699 requested a full tablet report, sending...
I20260811 06:19:07.244388 11489 ts_manager.cc:194] Registered new tserver with Master: cb5723272d51495fb045212da92fe130 (127.22.121.3:35683)
I20260811 06:19:07.245283 11489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:55501
I20260811 06:19:07.251823 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:19:07.273896 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.257978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.22.121.62:45699)
0811 06:19:07.258187 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.258192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.258732 (+ 540us) server_negotiation.cc:407] Connection header received
0811 06:19:07.258912 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.258915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.258989 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.259065 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:07.261055 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.261740 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.263739 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.264143 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.264233 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.267663 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:07.267668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:07.267671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:07.267700 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:07.270738 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.271780 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.271784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.271786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.271935 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.272411 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.272415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.272417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.272808 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:07.273086 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.273482 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.273648 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 06:19:07.278460 11489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:19:07.285447 11489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:19:07.336419 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.300174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:57980)
0811 06:19:07.321555 (+ 21381us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.321574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.321696 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.323995 (+ 2299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.324003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.324100 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.325840 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.325856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.328824 (+ 2968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.328830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.332785 (+ 3955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.332798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.335060 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.335068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.335368 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.335429 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.335559 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.335804 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21249,"mutex_wait_us":1,"spinlock_wait_cycles":12544,"thread_start_us":12083,"threads_started":1}
I20260811 06:19:07.337260 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.301067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.22.121.1:33265)
0811 06:19:07.323187 (+ 22120us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.323196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.323230 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:19:07.323359 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.323367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.323678 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.324380 (+ 702us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.326100 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.328648 (+ 2548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.332966 (+ 4318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.334776 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.334894 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.335237 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.336665 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.336880 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21915,"thread_start_us":22995,"threads_started":1}
I20260811 06:19:07.338536 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.299994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35683 (local address 127.0.0.1:36768)
0811 06:19:07.311761 (+ 11767us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.311804 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.312024 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.327258 (+ 15234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.327289 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.327316 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.328144 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.328156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.330455 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.330460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.333563 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.333579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.337648 (+ 4069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.337654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.337784 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.337816 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.337887 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.338007 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11411,"thread_start_us":22247,"threads_started":1}
I20260811 06:19:07.340946 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.313588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.22.121.2:38525)
0811 06:19:07.319827 (+ 6239us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.319837 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.322800 (+ 2963us) server_negotiation.cc:407] Connection header received
0811 06:19:07.322913 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.322922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.323251 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.323512 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.329085 (+ 5573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.331286 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.336897 (+ 5611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.339259 (+ 2362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.339449 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.339776 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.340407 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.340593 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6077,"thread_start_us":5929,"threads_started":1}
I20260811 06:19:07.342130 11758 tablet_service.cc:1467] Processing CreateTablet for tablet ac0babd53f304b0197fd5ba4bed640af (DEFAULT_TABLE table=test-table [id=5013186d39e44358bd4945f5e1adc389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:07.341114 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.319243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:38525 (local address 127.0.0.1:46530)
0811 06:19:07.322523 (+ 3280us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.322549 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.322675 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.323994 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.324003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.324049 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.324790 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.324808 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.331452 (+ 6644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.331457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.336518 (+ 5061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.336704 (+ 186us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.339993 (+ 3289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.340001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.340207 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.340241 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.340335 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.340457 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3137,"mutex_wait_us":25,"spinlock_wait_cycles":3840,"thread_start_us":2506,"threads_started":1}
I20260811 06:19:07.344094 11758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac0babd53f304b0197fd5ba4bed640af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:07.344463 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.300478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.22.121.3:35683)
0811 06:19:07.326737 (+ 26259us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.326743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.326769 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:19:07.326839 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.326843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.327058 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.327244 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.328421 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.330242 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.335359 (+ 5117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.337252 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.337381 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.343546 (+ 6165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.343645 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.343824 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26135,"thread_start_us":25853,"threads_started":1}
I20260811 06:19:07.347553 11986 tablet_service.cc:1467] Processing CreateTablet for tablet ac0babd53f304b0197fd5ba4bed640af (DEFAULT_TABLE table=test-table [id=5013186d39e44358bd4945f5e1adc389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:07.349382 11986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac0babd53f304b0197fd5ba4bed640af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:07.350107 12213 tablet_service.cc:1467] Processing CreateTablet for tablet ac0babd53f304b0197fd5ba4bed640af (DEFAULT_TABLE table=test-table [id=5013186d39e44358bd4945f5e1adc389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:07.353324 12213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac0babd53f304b0197fd5ba4bed640af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:07.371377 12360 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Bootstrap starting.
I20260811 06:19:07.374735 12362 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Bootstrap starting.
I20260811 06:19:07.378993 12360 tablet_bootstrap.cc:654] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:07.380205 12362 tablet_bootstrap.cc:654] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:07.381165 12360 log.cc:824] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:07.381601 12362 log.cc:824] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:07.388350 12364 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Bootstrap starting.
I20260811 06:19:07.391242 12362 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: No bootstrap required, opened a new log
I20260811 06:19:07.391592 12362 ts_tablet_manager.cc:1397] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.016s
I20260811 06:19:07.391739 12364 tablet_bootstrap.cc:654] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:07.393085 12364 log.cc:824] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:07.397125 12360 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: No bootstrap required, opened a new log
I20260811 06:19:07.397392 12360 ts_tablet_manager.cc:1397] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.012s
I20260811 06:19:07.402215 12364 tablet_bootstrap.cc:492] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: No bootstrap required, opened a new log
I20260811 06:19:07.402530 12364 ts_tablet_manager.cc:1397] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260811 06:19:07.403306 12362 raft_consensus.cc:348] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.403861 12362 raft_consensus.cc:374] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:07.404047 12362 raft_consensus.cc:729] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e6af1b230d446e8e72e8716f37bb87, State: Initialized, Role: FOLLOWER
I20260811 06:19:07.404840 12362 consensus_queue.cc:260] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.408322 12360 raft_consensus.cc:348] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.408818 12360 raft_consensus.cc:374] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:07.408955 12360 raft_consensus.cc:729] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86013ca7b13f4c8781a908d24ebe445d, State: Initialized, Role: FOLLOWER
I20260811 06:19:07.409741 12360 consensus_queue.cc:260] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.413051 12362 ts_tablet_manager.cc:1428] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Time spent starting tablet: real 0.021s user 0.013s sys 0.006s
I20260811 06:19:07.413180 12364 raft_consensus.cc:348] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.413468 12102 heartbeater.cc:503] Master 127.22.121.62:45699 was elected leader, sending a full tablet report...
I20260811 06:19:07.413664 12364 raft_consensus.cc:374] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:07.413820 12364 raft_consensus.cc:729] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb5723272d51495fb045212da92fe130, State: Initialized, Role: FOLLOWER
I20260811 06:19:07.414551 12364 consensus_queue.cc:260] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.417819 12360 ts_tablet_manager.cc:1428] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
I20260811 06:19:07.421305 12364 ts_tablet_manager.cc:1428] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Time spent starting tablet: real 0.019s user 0.012s sys 0.003s
I20260811 06:19:07.421700 12330 heartbeater.cc:503] Master 127.22.121.62:45699 was elected leader, sending a full tablet report...
W20260811 06:19:07.439134 11864 tablet.cc:2369] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:19:07.458380 12331 tablet.cc:2369] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:19:07.499356 12104 tablet.cc:2369] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:19:07.620013 12372 raft_consensus.cc:482] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:19:07.620308 12372 raft_consensus.cc:504] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.621942 12372 leader_election.cc:288] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86013ca7b13f4c8781a908d24ebe445d (127.22.121.1:33265), b4e6af1b230d446e8e72e8716f37bb87 (127.22.121.2:38525)
I20260811 06:19:07.627903 12370 raft_consensus.cc:482] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:19:07.628223 12370 raft_consensus.cc:504] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.628973 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.622253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.22.121.3:48207)
0811 06:19:07.622441 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.622462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.622581 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.623075 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.623094 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.623140 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.623383 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.623395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.624963 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.624968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.626700 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.626712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.628309 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.628314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.628472 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.628495 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.628585 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.628713 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:19:07.629053 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.622377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:48207 (local address 127.22.121.1:33265)
0811 06:19:07.622600 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.622606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.622627 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:19:07.622701 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.622706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.622783 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.622879 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.623545 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.624835 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.626850 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.628135 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.628293 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.628372 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.628705 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.628834 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:19:07.630254 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.622975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:59629 (local address 127.22.121.2:38525)
0811 06:19:07.623220 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.623227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.623725 (+ 498us) server_negotiation.cc:407] Connection header received
0811 06:19:07.623929 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.623933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.623988 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.624053 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.625028 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.626100 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.628263 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.629389 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.629558 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.629640 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.629907 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.630029 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:19:07.630369 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.622635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:38525 (local address 127.22.121.3:59629)
0811 06:19:07.623618 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.623637 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.623797 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.624090 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.624094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.624123 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.624872 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.624885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.626225 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.626229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.628093 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.628109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.629575 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.629582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.629742 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.629777 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.629878 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.629996 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":626,"threads_started":1}
I20260811 06:19:07.630187 11790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "cb5723272d51495fb045212da92fe130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86013ca7b13f4c8781a908d24ebe445d" is_pre_election: true
I20260811 06:19:07.630690 11790 raft_consensus.cc:2454] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb5723272d51495fb045212da92fe130 in term 0.
I20260811 06:19:07.630880 12370 leader_election.cc:288] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb5723272d51495fb045212da92fe130 (127.22.121.3:35683), b4e6af1b230d446e8e72e8716f37bb87 (127.22.121.2:38525)
I20260811 06:19:07.631799 12020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "cb5723272d51495fb045212da92fe130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4e6af1b230d446e8e72e8716f37bb87" is_pre_election: true
I20260811 06:19:07.632015 12139 leader_election.cc:302] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86013ca7b13f4c8781a908d24ebe445d, cb5723272d51495fb045212da92fe130; no voters:
I20260811 06:19:07.632390 12020 raft_consensus.cc:2454] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb5723272d51495fb045212da92fe130 in term 0.
I20260811 06:19:07.633458 12372 raft_consensus.cc:2781] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:19:07.633683 12372 raft_consensus.cc:482] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:19:07.633833 12372 raft_consensus.cc:3037] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:07.636850 12372 raft_consensus.cc:504] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.638055 11790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "cb5723272d51495fb045212da92fe130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86013ca7b13f4c8781a908d24ebe445d"
I20260811 06:19:07.638273 11790 raft_consensus.cc:3037] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:07.639808 12020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "cb5723272d51495fb045212da92fe130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4e6af1b230d446e8e72e8716f37bb87"
I20260811 06:19:07.640059 12020 raft_consensus.cc:3037] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:07.642766 11790 raft_consensus.cc:2454] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb5723272d51495fb045212da92fe130 in term 1.
I20260811 06:19:07.643230 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.631346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:49197 (local address 127.22.121.2:38525)
0811 06:19:07.632979 (+ 1633us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.632984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.633488 (+ 504us) server_negotiation.cc:407] Connection header received
0811 06:19:07.633710 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.633714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.633772 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.633825 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.635821 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.636802 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.639953 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.641588 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.641793 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.641868 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.642683 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.642821 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1542,"thread_start_us":3257,"threads_started":1}
I20260811 06:19:07.643312 12139 leader_election.cc:302] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86013ca7b13f4c8781a908d24ebe445d, cb5723272d51495fb045212da92fe130; no voters:
I20260811 06:19:07.643913 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.630763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:55217 (local address 127.22.121.3:35683)
0811 06:19:07.630925 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.630931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.631666 (+ 735us) server_negotiation.cc:407] Connection header received
0811 06:19:07.631920 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.631925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.631992 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.632055 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:07.632831 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.633817 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.636966 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.638338 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.638480 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.638547 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.643587 (+ 5040us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.643708 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:19:07.644865 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.630226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35683 (local address 127.22.121.1:55217)
0811 06:19:07.631569 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.631587 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.631763 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.632269 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.632273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.632299 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.632668 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.632682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.633980 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.633985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.636786 (+ 2801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.636803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.638900 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.638909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.643409 (+ 4500us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.643441 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.644087 (+ 646us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.644226 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":13,"thread_start_us":1280,"threads_started":1}
I20260811 06:19:07.644938 12383 raft_consensus.cc:2781] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:19:07.644224 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.631251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:38525 (local address 127.22.121.1:49197)
0811 06:19:07.633402 (+ 2151us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.633419 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:07.633585 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:07.633980 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:07.633985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:07.634010 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:07.635552 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.635567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.636969 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.636973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.639740 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:07.639755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.642204 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.642211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:07.642423 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.642460 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:19:07.643413 (+ 953us) client_negotiation.cc:240] Negotiation successful
0811 06:19:07.643538 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2018,"mutex_wait_us":632,"thread_start_us":1854,"threads_started":1}
I20260811 06:19:07.645938 12019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "86013ca7b13f4c8781a908d24ebe445d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4e6af1b230d446e8e72e8716f37bb87" is_pre_election: true
I20260811 06:19:07.645888 12248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac0babd53f304b0197fd5ba4bed640af" candidate_uuid: "86013ca7b13f4c8781a908d24ebe445d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb5723272d51495fb045212da92fe130" is_pre_election: true
I20260811 06:19:07.646313 12020 raft_consensus.cc:2454] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb5723272d51495fb045212da92fe130 in term 1.
I20260811 06:19:07.646644 12372 leader_election.cc:288] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [CANDIDATE]: Term 1 election: Requested vote from peers 86013ca7b13f4c8781a908d24ebe445d (127.22.121.1:33265), b4e6af1b230d446e8e72e8716f37bb87 (127.22.121.2:38525)
I20260811 06:19:07.648077 12383 raft_consensus.cc:686] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 1 LEADER]: Becoming Leader. State: Replica: cb5723272d51495fb045212da92fe130, State: Running, Role: LEADER
I20260811 06:19:07.648723 12383 consensus_queue.cc:237] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } }
I20260811 06:19:07.652938 11670 leader_election.cc:302] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86013ca7b13f4c8781a908d24ebe445d; no voters: b4e6af1b230d446e8e72e8716f37bb87, cb5723272d51495fb045212da92fe130
I20260811 06:19:07.653404 12370 raft_consensus.cc:2726] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:19:07.654237 11488 catalog_manager.cc:5495] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb5723272d51495fb045212da92fe130 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "cb5723272d51495fb045212da92fe130" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb5723272d51495fb045212da92fe130" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 } health_report { overall_health: UNKNOWN } } }
I20260811 06:19:07.698678 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.691316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.22.121.3:35683)
0811 06:19:07.691454 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.691458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.691747 (+ 289us) server_negotiation.cc:407] Connection header received
0811 06:19:07.691893 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.691896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.691967 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.692086 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:19:07.692535 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.693557 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.696023 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.696744 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.696831 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.697547 (+ 716us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:19:07.697993 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:19:07.698132 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.698214 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.698276 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:19:07.706763 12019 raft_consensus.cc:1261] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 1 FOLLOWER]: Refusing update from remote peer cb5723272d51495fb045212da92fe130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:19:07.706934 11790 raft_consensus.cc:1261] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 1 FOLLOWER]: Refusing update from remote peer cb5723272d51495fb045212da92fe130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:19:07.707824 12384 consensus_queue.cc:1035] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4e6af1b230d446e8e72e8716f37bb87" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 38525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:19:07.708297 12383 consensus_queue.cc:1035] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86013ca7b13f4c8781a908d24ebe445d" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:19:07.720990 12393 mvcc.cc:205] Tried to move back new op lower bound from 7317213788998389760 to 7317213788774182912. Current Snapshot: MvccSnapshot[applied={T|T < 7317213788998389760}]
I20260811 06:19:07.735343 11489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:19:07.735620 11489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:19:07.739975 11489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 569ace5ff60347159da6ab890db3c144: Sending DeleteTablet for 3 replicas of tablet ac0babd53f304b0197fd5ba4bed640af
I20260811 06:19:07.741189 12213 tablet_service.cc:1514] Processing DeleteTablet for tablet ac0babd53f304b0197fd5ba4bed640af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:07 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:36768
I20260811 06:19:07.741334 11986 tablet_service.cc:1514] Processing DeleteTablet for tablet ac0babd53f304b0197fd5ba4bed640af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:07 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:46530
I20260811 06:19:07.741334 11758 tablet_service.cc:1514] Processing DeleteTablet for tablet ac0babd53f304b0197fd5ba4bed640af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:07 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:57980
I20260811 06:19:07.743911 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 11630
I20260811 06:19:07.757618 12410 tablet_replica.cc:330] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: stopping tablet replica
I20260811 06:19:07.758558 12410 raft_consensus.cc:2229] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:19:07.759353 12408 tablet_replica.cc:330] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: stopping tablet replica
I20260811 06:19:07.759610 12412 tablet_replica.cc:330] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: stopping tablet replica
I20260811 06:19:07.759619 12410 raft_consensus.cc:2258] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:07.759910 12408 raft_consensus.cc:2229] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:19:07.760882 12408 raft_consensus.cc:2258] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:07.762140 12410 ts_tablet_manager.cc:1905] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:19:07.763003 12408 ts_tablet_manager.cc:1905] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:19:07 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:19:07.766813 12412 raft_consensus.cc:2229] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:19:07.772359 12410 ts_tablet_manager.cc:1918] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:19:07.772571 12410 log.cc:1196] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/ac0babd53f304b0197fd5ba4bed640af
I20260811 06:19:07.772816 12408 ts_tablet_manager.cc:1918] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:19:07.773205 12408 log.cc:1196] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/ac0babd53f304b0197fd5ba4bed640af
I20260811 06:19:07.773413 12410 ts_tablet_manager.cc:1939] T ac0babd53f304b0197fd5ba4bed640af P b4e6af1b230d446e8e72e8716f37bb87: Deleting consensus metadata
I20260811 06:19:07.773880 12408 ts_tablet_manager.cc:1939] T ac0babd53f304b0197fd5ba4bed640af P cb5723272d51495fb045212da92fe130: Deleting consensus metadata
I20260811 06:19:07.775626 11461 catalog_manager.cc:4841] TS b4e6af1b230d446e8e72e8716f37bb87 (127.22.121.2:38525): tablet ac0babd53f304b0197fd5ba4bed640af (table test-table [id=5013186d39e44358bd4945f5e1adc389]) successfully deleted
I20260811 06:19:07.775683 12412 raft_consensus.cc:2258] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:07.776845 11464 catalog_manager.cc:4841] TS cb5723272d51495fb045212da92fe130 (127.22.121.3:35683): tablet ac0babd53f304b0197fd5ba4bed640af (table test-table [id=5013186d39e44358bd4945f5e1adc389]) successfully deleted
I20260811 06:19:07.777380 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.745192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.22.121.1:33265)
0811 06:19:07.745366 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.745373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.749898 (+ 4525us) server_negotiation.cc:407] Connection header received
0811 06:19:07.750676 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.750682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.750794 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.750869 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:07.754485 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.755759 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.758702 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.759857 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.760018 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.766689 (+ 6671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:07.766701 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:07.766717 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:07.766760 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:07.772225 (+ 5465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.773264 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.773273 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.773276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.773531 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.774049 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.774054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.774058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.774724 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:07.775083 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.776474 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.776707 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:19:07.778203 11853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:07.778688 12412 ts_tablet_manager.cc:1905] T ac0babd53f304b0197fd5ba4bed640af P 86013ca7b13f4c8781a908d24ebe445d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:19:07.884364 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 11630
I20260811 06:19:07.916555 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 11876
W20260811 06:19:07.916535 11461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.121.1:33265 (error 108)
Aug 11 06:19:07 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:19:07.934253 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.927932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:57998)
0811 06:19:07.933987 (+ 6055us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.934103 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5951,"thread_start_us":4520,"threads_started":1}
W20260811 06:19:07.934513 11461 catalog_manager.cc:4584] TS 86013ca7b13f4c8781a908d24ebe445d (127.22.121.1:33265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac0babd53f304b0197fd5ba4bed640af: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
I20260811 06:19:07.937122 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.918119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.22.121.2:38525)
0811 06:19:07.918300 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:07.918306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:07.918325 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:19:07.918387 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:07.918392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:07.918499 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:07.918600 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:07.920479 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.921571 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.923601 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:07.924267 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:07.924385 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:07.930503 (+ 6118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:07.930522 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:07.930542 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:07.930596 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:07.934342 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.935217 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.935225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.935227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.935407 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:07.935807 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:07.935810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:07.935812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:07.936165 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:07.936397 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:07.936730 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:19:07.936900 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:19:07.937872 12093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:07.988464 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.987907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58010)
0811 06:19:07.988146 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.988271 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 06:19:07.989646 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:07.989223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58026)
0811 06:19:07.989390 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:07.989483 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:19:08.028031 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 11876
I20260811 06:19:08.050252 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.049704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58038)
0811 06:19:08.049945 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:08.050062 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:19:08.051415 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.050968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58048)
0811 06:19:08.051124 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:08.051214 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:19:08.054565 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 12112
Aug 11 06:19:08 dist-test-slave-gjrz krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:19:08.072290 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.055339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.22.121.3:35683)
0811 06:19:08.055483 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:08.055487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:08.055508 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:19:08.055692 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:08.055699 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:08.055787 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:08.055867 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:08.057797 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:08.058597 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:08.060640 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:08.061144 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:08.061240 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:08.066354 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:08.066366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:08.066381 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:08.066412 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:08.069582 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:08.070280 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:08.070287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:08.070289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:08.070412 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:08.070784 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:08.070788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:08.070790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:08.071184 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:08.071547 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:08.071772 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 06:19:08.071988 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:19:08.072899 12318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:08.132426 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.131985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58050)
0811 06:19:08.132183 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:08.132268 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:19:08.133489 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.133121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58066)
0811 06:19:08.133289 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:08.133361 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 06:19:08.193058 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 12112
I20260811 06:19:08.217720 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 11433
I20260811 06:19:08.218885 11543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:08.329685 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:08.328814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33265 (local address 127.0.0.1:58080)
0811 06:19:08.329389 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:08.329514 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421}
I20260811 06:19:08.330029 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 11433
I20260811 06:19:08.354812 23012 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:19:08.698770 23012 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:19:08.699074 23012 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:19:09.042953 23012 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:19:09.043 UTC [10693] LOG: received smart shutdown request
2026-08-11 06:19:09.047 UTC [10693] LOG: background worker "logical replication launcher" (PID 10713) exited with exit code 1
2026-08-11 06:19:09.047 UTC [10700] LOG: shutting down
2026-08-11 06:19:09.183 UTC [10693] LOG: database system is shut down
2026-08-11T06:19:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48797 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:19:09 dist-test-slave-gjrz krb5kdc[12441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:19:09 dist-test-slave-gjrz krb5kdc[12441](info): set up 2 sockets
Aug 11 06:19:09 dist-test-slave-gjrz krb5kdc[12441](info): commencing operation
krb5kdc: starting...
W20260811 06:19:10.514571 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:19:10 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:19:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangeradmin_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangeradmin_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangerlookup_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangerlookup_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
I20260811 06:19:10.803669 23012 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTjGG_n/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/postgres -l logfile start
2026-08-11 06:19:12.995 UTC [12488] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:19:12.995 UTC [12488] LOG: listening on IPv4 address "127.22.121.20", port 48849
2026-08-11 06:19:12.999 UTC [12488] LOG: listening on Unix socket "/tmp/.s.PGSQL.48849"
2026-08-11 06:19:13.020 UTC [12492] LOG: database system was shut down at 2026-08-11 06:19:11 UTC
2026-08-11 06:19:13.023 UTC [12488] LOG: database system is ready to accept connections
I20260811 06:19:14.147696 23012 mini_postgres.cc:96] Postgres bound to 48849
2026-08-11 06:19:14.167 UTC [12509] FATAL: database "slave" does not exist
127.22.121.20:48849 - accepting connections
I20260811 06:19:14.167687 23012 mini_ranger.cc:161] Starting Ranger...
I20260811 06:19:14.192734 23012 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:19:14.590683 23012 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:19:14.590786 23012 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin
2026-08-11 06:19:14,879 [I] DB FLAVOR :POSTGRES
2026-08-11 06:19:14,880 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:19:14,880 [I] Checking connection..
2026-08-11 06:19:14,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:19:15,134 [I] Checking connection passed.
2026-08-11 06:19:15,134 [I] --------- Verifying version history table ---------
2026-08-11 06:19:15,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:19:15,386 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:19:15,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:19:15,622 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:19:15,622 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:19:15,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:19:15.868 UTC [12629] WARNING: there is no transaction in progress
2026-08-11 06:19:15,874 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:19:15,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:19:16,110 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:19:16,110 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:19:16,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:19:16,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:19:16,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-11 06:19:16,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:19:17,025 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:19:17,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:19:17,278 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:19:17,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:19:17,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:19:17,787 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:19:17,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:19:18.666 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.681 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.695 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.708 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.722 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.737 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.785 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.791 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.803 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.812 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.821 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.832 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.841 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.846 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.852 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:18.857 UTC [12812] WARNING: there is no transaction in progress
2026-08-11 06:19:19,413 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:19:19,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:19:19,649 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:19:19,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:19:19,878 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:19:19,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:19:20,131 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:19:20,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:19:20,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:19:20,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-11 06:19:20,836 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:19:20,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:19:21,077 [I] DB_PATCHES have already been applied
I20260811 06:19:21.083086 23012 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTjGG_n/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:19:21.083168 23012 mini_ranger.cc:193] Using host: 127.22.121.20
I20260811 06:19:21.099536 23012 mini_ranger.cc:227] Ranger admin URL: http://127.22.121.20:58949
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:19:22.395646 23012 mini_ranger.cc:160] Time spent starting Ranger: real 8.228s user 0.012s sys 0.031s
Aug 11, 2026 6:19:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:19:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 6:19:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:19:41 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:19:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:19:43.173369 23012 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.121.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.121.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangerkms_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.121.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/rangerkms_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.121.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.121.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.121.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/HTTP_127.22.121.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:19:43 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429183, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:19:43.341167 23012 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:19:43.362536 23012 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:19:44.121192 23012 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:19:44.121311 23012 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms
I20260811 06:19:44.121349 23012 mini_ranger_kms.cc:227] Using postgres at 127.22.121.20:48849
2026-08-11 06:19:44,513 [I] DB FLAVOR :POSTGRES
2026-08-11 06:19:44,514 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:19:44,514 [I] Checking connection
2026-08-11 06:19:44,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:19:44,764 [I] connection success
2026-08-11 06:19:44,764 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:19:44,764 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:19:44,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:19:44,987 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:19:44,987 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:19:44,987 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:19:44,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.121.20:48849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:19:45,246 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:19:45.705964 23012 mini_ranger_kms.cc:310] Created kms service
I20260811 06:19:45.898170 23012 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:19:45.951435 23012 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:19:45 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429183, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:46.344515 23012 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:19:46.344833 23012 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTjGG_n/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTjGG_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:19:46.344888 23012 mini_ranger_kms.cc:240] Using host: 127.22.121.20
I20260811 06:19:46.364565 23012 mini_ranger_kms.cc:276] Ranger KMS PID: 13118
I20260811 06:19:46.364688 23012 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.121.20:55825
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:19:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:19:48 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429188, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:48.945916 23012 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.605s user 0.061s sys 0.359s
I20260811 06:19:48.946143 23012 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:19:48.946321 23012 mini_ranger_kms.cc:401] 127.22.121.20:55825/kms/v1/keys
Aug 11 06:19:49 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429188, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
Aug 11 06:19:49 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429188, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.121.20@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
Aug 11 06:19:51 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:19:51.522622 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40671
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:40671
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:19:51.832867 13201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:51.833118 13201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:51.833243 13201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:51.840037 13201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:19:51.840132 13201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:51.840211 13201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:51.840282 13201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:19:51.840346 13201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:19:51.849795 13201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.121.20:55825/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:40671
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:51.850821 13201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:51.852139 13201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:51.863302 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:51.863298 13212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:51.863324 13213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:51.863848 13201 server_base.cc:1034] running on GCE node
I20260811 06:19:51.864527 13201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:51.865694 13201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:51.866916 13201 hybrid_clock.cc:648] HybridClock initialized: now 1786429191866850 us; error 37 us; skew 500 ppm
I20260811 06:19:51.867244 13201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:51 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:51.873070 13201 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:19:51.874382 13201 webserver.cc:460] Webserver started at http://127.22.121.62:40249/ using document root <none> and password file <none>
I20260811 06:19:51.874962 13201 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:51.875066 13201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:51.875378 13201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:51 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:51.922986 13201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "1d14ddf4b2614e52a52ee1873ec825e8"
format_stamp: "Formatted at 2026-08-11 06:19:51 on dist-test-slave-gjrz"
server_key: "009207ae22a00f38150a6fd39b7fc034"
server_key_iv: "cadf6c7283114f7627c6cf2ff0b608c0"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:51.923900 13201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "1d14ddf4b2614e52a52ee1873ec825e8"
format_stamp: "Formatted at 2026-08-11 06:19:51 on dist-test-slave-gjrz"
server_key: "009207ae22a00f38150a6fd39b7fc034"
server_key_iv: "cadf6c7283114f7627c6cf2ff0b608c0"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:51.928964 13201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:19:51.943773 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:51.945093 13201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:19:51.945261 13201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "1d14ddf4b2614e52a52ee1873ec825e8"
format_stamp: "Formatted at 2026-08-11 06:19:51 on dist-test-slave-gjrz"
server_key: "009207ae22a00f38150a6fd39b7fc034"
server_key_iv: "cadf6c7283114f7627c6cf2ff0b608c0"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:51.945451 13201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:51.964911 13201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:51.968539 13201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:51.968806 13201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:51.985602 13201 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:40671
I20260811 06:19:51.985633 13334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:40671 every 8 connection(s)
I20260811 06:19:51.986796 13201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:19:51.990262 13335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:51.993757 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 13201
I20260811 06:19:51.993942 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:19:51.994558 23012 external_mini_cluster.cc:1428] Setting key 2ab82d84088a25123f2045f9b155ea1e
I20260811 06:19:52.000743 13335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Bootstrap starting.
I20260811 06:19:52.004135 13335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:52.005244 13335 log.cc:824] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:52.008440 13335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: No bootstrap required, opened a new log
Aug 11 06:19:52 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:52.018054 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:51.996401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.22.121.62:40671)
0811 06:19:51.997131 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:51.997143 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:51.997180 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:19:51.999193 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:51.999214 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:51.999615 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:51.999902 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:52.001571 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:52.002860 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:52.004682 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:52.005337 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:52.005487 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:52.009908 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:52.009922 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:52.009940 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:52.009988 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:52.013838 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:52.014483 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:52.014496 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:52.014505 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:52.014686 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:52.014994 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:52.014999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:52.015002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:52.015713 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:52.015999 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:52.016384 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 06:19:52.016780 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":343,"threads_started":1}
I20260811 06:19:52.018500 13335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:52.018839 13335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:52.018944 13335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d14ddf4b2614e52a52ee1873ec825e8, State: Initialized, Role: FOLLOWER
I20260811 06:19:52.019599 13335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:52.019894 13335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:19:52.020032 13335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:19:52.020215 13335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:52.022413 13335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:52.022954 13335 leader_election.cc:302] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d14ddf4b2614e52a52ee1873ec825e8; no voters:
I20260811 06:19:52.023533 13335 leader_election.cc:288] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:19:52.023630 13345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:19:52.024281 13345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1d14ddf4b2614e52a52ee1873ec825e8, State: Running, Role: LEADER
I20260811 06:19:52.024770 13345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:52.025372 13335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:19:52.027436 13350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d14ddf4b2614e52a52ee1873ec825e8. Latest consensus state: current_term: 1 leader_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } } }
I20260811 06:19:52.027798 13350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:52.028934 13346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } } }
I20260811 06:19:52.029251 13346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:52.029855 13357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:19:52.035212 13357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:19:52.036019 13335 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:19:52.061450 13357 catalog_manager.cc:1347] Generated new cluster ID: 4a1949d3c72d4775be169c971bf0b37b
I20260811 06:19:52.061606 13357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:19:52.076233 13357 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:19:52.078202 13357 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:19:52.099004 13357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Generated new TSK 0
I20260811 06:19:52.100302 13357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:19:52.837172 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:19:53.220741 13415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:53.220980 13415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:53.221095 13415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:53.228950 13415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:53.229065 13415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:53.229238 13415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:19:53.239651 13415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:53.241106 13415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:53.242734 13415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:53.258997 13432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:53.259027 13433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:53.267104 13415 server_base.cc:1034] running on GCE node
W20260811 06:19:53.267455 13437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:53.268029 13415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:53.268973 13415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:53.270221 13415 hybrid_clock.cc:648] HybridClock initialized: now 1786429193270173 us; error 40 us; skew 500 ppm
I20260811 06:19:53.270524 13415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:53.274197 13415 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:19:53.280468 13415 webserver.cc:460] Webserver started at http://127.22.121.1:43643/ using document root <none> and password file <none>
I20260811 06:19:53.281160 13415 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:53.281306 13415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:53.281625 13415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:53.302351 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "dc86bb9fd86d4119ac9755c15477519a"
format_stamp: "Formatted at 2026-08-11 06:19:53 on dist-test-slave-gjrz"
server_key: "da93bfb67da3414f54b6bf1c3ef3c823"
server_key_iv: "ee185ee1678ffc3729386074d639f66c"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:53.303376 13415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "dc86bb9fd86d4119ac9755c15477519a"
format_stamp: "Formatted at 2026-08-11 06:19:53 on dist-test-slave-gjrz"
server_key: "da93bfb67da3414f54b6bf1c3ef3c823"
server_key_iv: "ee185ee1678ffc3729386074d639f66c"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:53.312376 13415 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260811 06:19:53.334870 13445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:53.336412 13415 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260811 06:19:53.336597 13415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "dc86bb9fd86d4119ac9755c15477519a"
format_stamp: "Formatted at 2026-08-11 06:19:53 on dist-test-slave-gjrz"
server_key: "da93bfb67da3414f54b6bf1c3ef3c823"
server_key_iv: "ee185ee1678ffc3729386074d639f66c"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:53.336818 13415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:53.361490 13415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:53.366176 13415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:53.366513 13415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:53.367584 13415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:53.369153 13415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:53.369239 13415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:53.369345 13415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:53.369398 13415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:53.415349 13415 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33035
I20260811 06:19:53.415388 13644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33035 every 8 connection(s)
I20260811 06:19:53.416967 13415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:19:53.426455 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 13415
I20260811 06:19:53.426657 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:19:53.427542 23012 external_mini_cluster.cc:1428] Setting key f0b9959c57896b657e9c953614d9e209
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:53.469697 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:53.427172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:44627 (local address 127.22.121.62:40671)
0811 06:19:53.438910 (+ 11738us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:53.438917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:53.438949 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:19:53.439008 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:53.439014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:53.439101 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:53.439218 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:53.450348 (+ 11130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:53.452953 (+ 2605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:53.456296 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:53.456972 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:53.457073 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:53.462404 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:53.462410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:53.462414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:53.462464 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:53.465831 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:53.466783 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:53.466789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:53.466793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:53.466958 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:53.467550 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:53.467555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:53.467559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:53.467927 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:53.468073 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:53.469089 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0811 06:19:53.469314 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11609,"thread_start_us":11484,"threads_started":1}
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
I20260811 06:19:53.472051 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:53.426510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.1:44627)
0811 06:19:53.433510 (+ 7000us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:53.433570 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:53.436163 (+ 2593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:53.447624 (+ 11461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:53.447637 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:53.448325 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:53.449225 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:53.449254 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:53.453182 (+ 3928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:53.453188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:53.456120 (+ 2932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:53.456138 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:53.457428 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:53.457436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:53.457721 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:53.458679 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:53.458712 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:53.462140 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:53.466017 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:53.466028 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:53.466046 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:53.466634 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:53.467123 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:53.467128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:53.467131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:53.467407 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:53.468177 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:53.468186 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:53.468612 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 06:19:53.470243 (+ 1631us) client_negotiation.cc:240] Negotiation successful
0811 06:19:53.470790 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6676,"thread_start_us":6436,"threads_started":1}
I20260811 06:19:53.473742 13646 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:53.474042 13646 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:53.474707 13646 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
I20260811 06:19:53.476974 13278 ts_manager.cc:194] Registered new tserver with Master: dc86bb9fd86d4119ac9755c15477519a (127.22.121.1:33035)
I20260811 06:19:53.478827 13278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:44627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:19:53.508203 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
Aug 11 06:19:53 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429193, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
W20260811 06:19:53.957083 13657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:53.957744 13657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:53.958026 13657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:53.975297 13657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:53.975488 13657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:53.975718 13657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:19:54.001047 13657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:54.003062 13657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:54.004897 13657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:54.049782 13670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:54.051875 13669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:54.063023 13657 server_base.cc:1034] running on GCE node
W20260811 06:19:54.065251 13674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:54.066094 13657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:54.075526 13657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:54.076771 13657 hybrid_clock.cc:648] HybridClock initialized: now 1786429194076717 us; error 38 us; skew 500 ppm
I20260811 06:19:54.077191 13657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:54 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:54.081566 13657 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:19:54.114706 13657 webserver.cc:460] Webserver started at http://127.22.121.2:43189/ using document root <none> and password file <none>
I20260811 06:19:54.115859 13657 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:54.116102 13657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:54.116531 13657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:54 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:54.156213 13657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "dfaaf38158e845aa9dbaad8d40991b2a"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "a420167fac717eb43a905c48169b76cc"
server_key_iv: "2536463959426e6f7432620f65193bcd"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.157272 13657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "dfaaf38158e845aa9dbaad8d40991b2a"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "a420167fac717eb43a905c48169b76cc"
server_key_iv: "2536463959426e6f7432620f65193bcd"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.172456 13657 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.008s sys 0.002s
I20260811 06:19:54.228439 13682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:54.230381 13657 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.004s sys 0.002s
I20260811 06:19:54.230669 13657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "dfaaf38158e845aa9dbaad8d40991b2a"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "a420167fac717eb43a905c48169b76cc"
server_key_iv: "2536463959426e6f7432620f65193bcd"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.231048 13657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:54.296990 13657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:54.302241 13657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:54.302641 13657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:54.310109 13657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:54.318229 13657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:54.318389 13657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:19:54.318564 13657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:54.318656 13657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:54.423460 13657 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:39053
I20260811 06:19:54.423517 13883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:39053 every 8 connection(s)
I20260811 06:19:54.425078 13657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:19:54.430965 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 13657
I20260811 06:19:54.431125 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:19:54.431813 23012 external_mini_cluster.cc:1428] Setting key 8e0a3c55865b549e10ba76623cb15ce6
Aug 11 06:19:54 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
I20260811 06:19:54.477391 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:54.429373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:36261 (local address 127.22.121.62:40671)
0811 06:19:54.451699 (+ 22326us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:54.451706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:54.455084 (+ 3378us) server_negotiation.cc:407] Connection header received
0811 06:19:54.455155 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:54.455159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:54.455246 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:54.455371 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:54.457535 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:54.458708 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:54.461107 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:54.461706 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:54.461799 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:54.466613 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:54.466618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:54.466621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:54.466654 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:54.469418 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:54.470141 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:54.470145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:54.470147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:54.470285 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:54.475433 (+ 5148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:54.475438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:54.475441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:54.475778 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:54.475913 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:54.476829 (+ 916us) server_negotiation.cc:299] Negotiation successful
0811 06:19:54.477060 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22183,"thread_start_us":22072,"threads_started":1}
I20260811 06:19:54.479792 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:54.428799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.2:36261)
0811 06:19:54.452293 (+ 23494us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:54.452348 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:54.454699 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:54.455876 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:54.455889 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:54.456544 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:54.457332 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:54.457358 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:54.458918 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:54.458923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:54.460938 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:54.460953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:54.462164 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:54.462171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:54.462440 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:54.463549 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:54.463585 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:54.466382 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:54.469558 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:54.469567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:54.469582 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:54.470021 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:54.470665 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:54.470669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:54.470672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:54.470906 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:54.475998 (+ 5092us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:54.476005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:54.476412 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 06:19:54.477900 (+ 1488us) client_negotiation.cc:240] Negotiation successful
0811 06:19:54.478415 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23172,"thread_start_us":22947,"threads_started":1}
I20260811 06:19:54.481626 13884 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:54.481997 13884 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:54.482805 13884 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:54.483587 13646 heartbeater.cc:503] Master 127.22.121.62:40671 was elected leader, sending a full tablet report...
I20260811 06:19:54.486440 13278 ts_manager.cc:194] Registered new tserver with Master: dfaaf38158e845aa9dbaad8d40991b2a (127.22.121.2:39053)
I20260811 06:19:54.487488 13278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:36261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:19:54.500361 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:19:54.816828 13894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:54.817088 13894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:54.817221 13894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:54.824170 13894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:54.824273 13894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:54.824414 13894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:19:54.833658 13894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:54.834769 13894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:54.836000 13894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:54.848274 13903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:54.848435 13902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:54.848462 13906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:54.849335 13894 server_base.cc:1034] running on GCE node
I20260811 06:19:54.849877 13894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:54.850771 13894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:54.851962 13894 hybrid_clock.cc:648] HybridClock initialized: now 1786429194851901 us; error 43 us; skew 500 ppm
I20260811 06:19:54.852239 13894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:54 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:54.855084 13894 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:19:54.856416 13894 webserver.cc:460] Webserver started at http://127.22.121.3:43903/ using document root <none> and password file <none>
I20260811 06:19:54.857093 13894 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:54.857220 13894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:19:54.857488 13894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:19:54 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:54.871093 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "cbd6977f2e6e47dab5c29d883d491368"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "f85795cd4a8365ab365c01b0b7c9b3fe"
server_key_iv: "496ba2c594565085a5131910efbd1e6a"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.871910 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "cbd6977f2e6e47dab5c29d883d491368"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "f85795cd4a8365ab365c01b0b7c9b3fe"
server_key_iv: "496ba2c594565085a5131910efbd1e6a"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.877395 13894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:19:54.894253 13915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:54.895586 13894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:19:54.895722 13894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "cbd6977f2e6e47dab5c29d883d491368"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "f85795cd4a8365ab365c01b0b7c9b3fe"
server_key_iv: "496ba2c594565085a5131910efbd1e6a"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:54.895991 13894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:54.915349 13894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:54.920394 13894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:54.920987 13894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:54.922212 13894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:54.924235 13894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:54.924364 13894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:54.924582 13894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:54.924691 13894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:54.994580 13894 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41733
I20260811 06:19:54.994711 14097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41733 every 8 connection(s)
I20260811 06:19:54.995980 13894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:19:55.002664 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 13894
I20260811 06:19:55.002820 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:19:55.003441 23012 external_mini_cluster.cc:1428] Setting key d27dbfe760a94f811c762b9a9de399d4
Aug 11 06:19:55 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429194, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:55.021335 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:54.998972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:47741 (local address 127.22.121.62:40671)
0811 06:19:54.999727 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:54.999734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:55.001096 (+ 1362us) server_negotiation.cc:407] Connection header received
0811 06:19:55.003611 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:55.003617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:55.003770 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:55.003910 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:55.005982 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.007044 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.008996 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.009700 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.009812 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:55.014104 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:55.014110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:55.014113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:55.014145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:55.017872 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.018623 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.018628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.018630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.018845 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.019432 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.019436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.019438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.019728 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:55.019899 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:55.020537 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 06:19:55.020788 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":507,"threads_started":1}
I20260811 06:19:55.022439 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:54.999537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.3:47741)
0811 06:19:55.000908 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:55.000963 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:55.003311 (+ 2348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:55.004146 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:55.004157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:55.004801 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:55.005819 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:55.005841 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.007178 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.007182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:55.008865 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:55.008878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.009983 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.009992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:55.010293 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:55.010994 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:55.011015 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:55.013873 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:55.017998 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:55.018007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:55.018023 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:55.018498 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:55.018955 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:55.018959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:55.018962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:55.019292 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:55.019855 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:55.019863 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:55.020194 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 06:19:55.020424 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 06:19:55.021026 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":743,"threads_started":1}
I20260811 06:19:55.023936 14098 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:55.024266 14098 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:55.025193 14098 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:55.026916 13278 ts_manager.cc:194] Registered new tserver with Master: cbd6977f2e6e47dab5c29d883d491368 (127.22.121.3:41733)
I20260811 06:19:55.027896 13278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:47741
I20260811 06:19:55.031142 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:19:55.032912 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 13415
Aug 11 06:19:55 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:19:55.052328 14104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:55.034064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56408 (local address 127.22.121.1:33035)
0811 06:19:55.035188 (+ 1124us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:55.035195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:55.035219 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:19:55.035314 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:55.035324 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:55.035533 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:55.035649 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:55.037988 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.039571 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.041040 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.041704 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.041824 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:55.046517 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:55.046530 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:55.046541 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:55.046576 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:55.049739 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.050598 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.050605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.050607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.050751 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.051070 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.051073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.051075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.051409 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:55.051643 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:55.051880 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:19:55.052063 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":715,"threads_started":1}
I20260811 06:19:55.053500 13635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:55.135911 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 13415
I20260811 06:19:55.160513 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 13657
Aug 11 06:19:55 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:19:55.175590 14109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:55.161371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.22.121.2:39053)
0811 06:19:55.161879 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:55.161898 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:55.161924 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:19:55.161987 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:55.161993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:55.162203 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:55.162341 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:55.163587 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.164588 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.165906 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.166440 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.166533 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:55.170661 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:55.170673 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:55.170687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:55.170718 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:55.173253 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.173831 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.173837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.173839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.173972 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.174246 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.174249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.174251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.174559 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:55.174788 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:55.174994 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 06:19:55.175189 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":246,"threads_started":1}
I20260811 06:19:55.176769 13878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:55.256364 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 13657
I20260811 06:19:55.276867 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 13894
Aug 11 06:19:55 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:19:55.292361 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:55.277911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.22.121.3:41733)
0811 06:19:55.278568 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:55.278576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:55.278607 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:19:55.278667 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:55.278673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:55.278884 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:55.279008 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:55.280250 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.281258 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.282538 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:55.283088 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:55.283182 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:55.287308 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:55.287323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:55.287341 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:55.287379 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:55.289922 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.290506 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.290512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.290514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.290635 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:55.290979 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:55.290982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:55.290984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:55.291355 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:55.291603 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:55.291814 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:19:55.292007 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
I20260811 06:19:55.293642 14088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:55.374181 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 13894
I20260811 06:19:55.399674 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 13201
I20260811 06:19:55.400872 13328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:55.504891 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 13201
I20260811 06:19:55.537708 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40671
--webserver_interface=127.22.121.62
--webserver_port=40249
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:40671
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:19:55.845822 14119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:55.846079 14119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:55.846197 14119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:55.853058 14119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:19:55.853160 14119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:55.853235 14119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:55.853314 14119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:19:55.853379 14119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:19:55.862751 14119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:40671
--ranger_config_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:55.863807 14119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:55.864981 14119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:55.875607 14127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:55.875636 14134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:55.875669 14129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:55.876756 14119 server_base.cc:1034] running on GCE node
I20260811 06:19:55.877413 14119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:55.878578 14119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:55.879766 14119 hybrid_clock.cc:648] HybridClock initialized: now 1786429195879715 us; error 44 us; skew 500 ppm
I20260811 06:19:55.880069 14119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:55 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429195, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:55.882943 14119 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:19:55.884636 14119 webserver.cc:460] Webserver started at http://127.22.121.62:40249/ using document root <none> and password file <none>
I20260811 06:19:55.885222 14119 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:55.885336 14119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:19:56 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429195, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:56.043433 14119 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:19:56.046310 14143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:56.047655 14119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:19:56.047808 14119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "1d14ddf4b2614e52a52ee1873ec825e8"
format_stamp: "Formatted at 2026-08-11 06:19:51 on dist-test-slave-gjrz"
server_key: "009207ae22a00f38150a6fd39b7fc034"
server_key_iv: "cadf6c7283114f7627c6cf2ff0b608c0"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:56.048751 14119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:56.070973 14119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:56.074529 14119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:56.074774 14119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:56.091336 14119 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:40671
I20260811 06:19:56.091365 14240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:40671 every 8 connection(s)
I20260811 06:19:56.092674 14119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:19:56.098980 14241 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:19:56.099692 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 14119
I20260811 06:19:56.100454 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:33035
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:40671
--webserver_port=43643
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:19:56.102222 14241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Bootstrap starting.
I20260811 06:19:56.119848 14241 log.cc:824] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:56.128376 14241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:19:56.128849 14241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Bootstrap complete.
I20260811 06:19:56.137483 14241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:56.138325 14241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d14ddf4b2614e52a52ee1873ec825e8, State: Initialized, Role: FOLLOWER
I20260811 06:19:56.138944 14241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:56.139160 14241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:19:56.139328 14241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:19:56.139489 14241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:19:56.143006 14241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:56.143481 14241 leader_election.cc:302] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d14ddf4b2614e52a52ee1873ec825e8; no voters:
I20260811 06:19:56.144025 14241 leader_election.cc:288] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:19:56.144145 14250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:19:56.145215 14250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [term 2 LEADER]: Becoming Leader. State: Replica: 1d14ddf4b2614e52a52ee1873ec825e8, State: Running, Role: LEADER
I20260811 06:19:56.145788 14250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } }
I20260811 06:19:56.146062 14241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:19:56.148146 14252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } } }
I20260811 06:19:56.148525 14252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:56.150727 14254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d14ddf4b2614e52a52ee1873ec825e8. Latest consensus state: current_term: 2 leader_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d14ddf4b2614e52a52ee1873ec825e8" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40671 } } }
I20260811 06:19:56.151121 14254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:19:56.153190 14261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:19:56.158784 14261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:19:56.160308 14261 catalog_manager.cc:1259] Loaded cluster ID: 4a1949d3c72d4775be169c971bf0b37b
I20260811 06:19:56.160501 14261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:19:56.164660 14261 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:19:56.166224 14261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Loaded TSK: 0
I20260811 06:19:56.167850 14261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:19:56.180166 14241 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 06:19:56.532655 14246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:56.533002 14246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:56.533252 14246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:56.544339 14246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:56.544512 14246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:56.544752 14246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:19:56.559473 14246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:33035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=43643
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:56.561185 14246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:56.562930 14246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:56.610985 14305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:56.614699 14304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:56.642246 14246 server_base.cc:1034] running on GCE node
W20260811 06:19:56.643077 14308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:56.643981 14246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:56.663527 14246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:56.665946 14246 hybrid_clock.cc:648] HybridClock initialized: now 1786429196665846 us; error 72 us; skew 500 ppm
I20260811 06:19:56.666450 14246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:56 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429196, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:56.672905 14246 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:19:56.678454 14246 webserver.cc:460] Webserver started at http://127.22.121.1:43643/ using document root <none> and password file <none>
I20260811 06:19:56.679457 14246 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:56.679698 14246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:19:56 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429196, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:56.712128 14246 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:19:56.714977 14322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:56.716375 14246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:19:56.716539 14246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "dc86bb9fd86d4119ac9755c15477519a"
format_stamp: "Formatted at 2026-08-11 06:19:53 on dist-test-slave-gjrz"
server_key: "da93bfb67da3414f54b6bf1c3ef3c823"
server_key_iv: "ee185ee1678ffc3729386074d639f66c"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:56.717504 14246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:56.740237 14246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:56.744184 14246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:56.744498 14246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:56.745683 14246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:56.747242 14246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:56.747367 14246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:19:56.747465 14246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:56.747576 14246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:56.809269 14246 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33035
I20260811 06:19:56.809321 14514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33035 every 8 connection(s)
I20260811 06:19:56.810868 14246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:19:56.819584 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 14246
I20260811 06:19:56.820369 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:39053
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:40671
--webserver_port=43189
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:56 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429196, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:56.842682 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:56.815198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:56785 (local address 127.22.121.62:40671)
0811 06:19:56.817061 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:56.817081 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:56.818239 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 06:19:56.820751 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:56.820774 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:56.821269 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:56.821669 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:56.824240 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:56.826237 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:56.828563 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:56.829498 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:56.829636 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:56.833858 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:56.833868 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:56.833885 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:56.833943 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:56.837485 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:56.838378 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:56.838387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:56.838395 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:56.838604 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:56.839133 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:56.839138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:56.839141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:56.839762 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:56.839940 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:56.840938 (+ 998us) server_negotiation.cc:299] Negotiation successful
0811 06:19:56.841345 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1576,"thread_start_us":1338,"threads_started":1}
I20260811 06:19:56.845057 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:56.815368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.1:56785)
0811 06:19:56.818050 (+ 2682us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:56.818103 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:56.820037 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:56.822536 (+ 2499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:56.822546 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:56.823198 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:56.823994 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:56.824021 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:56.826476 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:56.826482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:56.828401 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:56.828418 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:56.830036 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:56.830043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:56.830284 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:56.831041 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:56.831076 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:56.833604 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:56.837639 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:56.837647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:56.837660 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:56.838200 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:56.838755 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:56.838760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:56.838763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:56.839009 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:56.840029 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:56.840036 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:56.840373 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 06:19:56.843396 (+ 3023us) client_negotiation.cc:240] Negotiation successful
0811 06:19:56.843860 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2397,"thread_start_us":1973,"threads_started":1}
I20260811 06:19:56.848711 14516 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:56.849027 14516 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:56.849888 14516 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
W20260811 06:19:56.853404 14516 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:40671 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:19:56.853616 14516 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:56.853847 14516 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:56.855052 14516 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:56.855365 14516 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
W20260811 06:19:56.856401 14516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:19:57.253017 14522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:57.253436 14522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:57.253648 14522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:57.264994 14522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:57.265177 14522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:57.265394 14522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:19:57.280191 14522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=43189
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:57.281930 14522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:57.283568 14522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:57.309981 14545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:57.310417 14547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:57.319247 14550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:57.319758 14522 server_base.cc:1034] running on GCE node
I20260811 06:19:57.320534 14522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:57.321926 14522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:57.323220 14522 hybrid_clock.cc:648] HybridClock initialized: now 1786429197323115 us; error 106 us; skew 500 ppm
I20260811 06:19:57.323648 14522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:57.330408 14522 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:19:57.332367 14522 webserver.cc:460] Webserver started at http://127.22.121.2:43189/ using document root <none> and password file <none>
I20260811 06:19:57.333335 14522 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:57.333518 14522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:57.368381 14522 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.008s sys 0.003s
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:57.374125 14567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:57.375877 14522 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260811 06:19:57.376048 14522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "dfaaf38158e845aa9dbaad8d40991b2a"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "a420167fac717eb43a905c48169b76cc"
server_key_iv: "2536463959426e6f7432620f65193bcd"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:57.377388 14522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:19:57.402007 14522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:57.406955 14522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:57.407317 14522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:57.410316 14522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:57.411640 14522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:57.411737 14522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:57.411834 14522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:57.411911 14522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:57.452171 14522 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:39053
I20260811 06:19:57.452253 14773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:39053 every 8 connection(s)
I20260811 06:19:57.454057 14522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:19:57.463865 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 14522
I20260811 06:19:57.464609 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.121.20:55825/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:41733
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:40671
--webserver_port=43903
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:57.506950 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:57.457562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.2:55485)
0811 06:19:57.475349 (+ 17787us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:57.475416 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:57.477598 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:57.478492 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:57.478503 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:57.479061 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:57.479855 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:57.479880 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:57.481390 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:57.481395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:57.483447 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:57.483462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:57.484552 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:57.484558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:57.484816 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:57.485606 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:57.485633 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:57.498232 (+ 12599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:57.502319 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:57.502329 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:57.502352 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:57.502941 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:57.503429 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:57.503434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:57.503438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:57.503704 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:57.504393 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:57.504401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:57.504795 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 06:19:57.505094 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:19:57.505653 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17446,"thread_start_us":17237,"threads_started":1}
I20260811 06:19:57.508251 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:57.458140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55485 (local address 127.22.121.62:40671)
0811 06:19:57.474674 (+ 16534us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:57.474681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:57.475929 (+ 1248us) server_negotiation.cc:407] Connection header received
0811 06:19:57.477873 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:57.477879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:57.477964 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:57.478077 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:57.480083 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:57.481175 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:57.483625 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:57.484300 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:57.484438 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:57.498510 (+ 14072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:57.498518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:57.498523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:57.498567 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:57.502143 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:57.503098 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:57.503104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:57.503108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:57.503302 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:57.503831 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:57.503836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:57.503840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:57.504190 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:57.504323 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:57.507657 (+ 3334us) server_negotiation.cc:299] Negotiation successful
0811 06:19:57.507881 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16406,"thread_start_us":16525,"threads_started":1}
I20260811 06:19:57.509738 14774 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:57.510059 14774 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:57.510948 14774 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:57.513046 14181 ts_manager.cc:194] Registered new tserver with Master: dfaaf38158e845aa9dbaad8d40991b2a (127.22.121.2:39053)
I20260811 06:19:57.515038 14181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:55485
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:57.856567 14516 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:57.857025 14516 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:57.858203 14181 ts_manager.cc:194] Registered new tserver with Master: dc86bb9fd86d4119ac9755c15477519a (127.22.121.1:33035)
I20260811 06:19:57.859504 14181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:56785
W20260811 06:19:57.908926 14779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:19:57.909253 14779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:19:57.909451 14779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:19:57.920373 14779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:19:57.920545 14779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:19:57.920742 14779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:19:57.935333 14779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.121.20:55825/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:41733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=43903
--tserver_master_addrs=127.22.121.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:19:57.936939 14779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:19:57.938592 14779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:19:57.954891 14790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:19:57.955549 14791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:57.959527 14779 server_base.cc:1034] running on GCE node
W20260811 06:19:57.960342 14795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:19:57.961117 14779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:19:57.962301 14779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:19:57.963574 14779 hybrid_clock.cc:648] HybridClock initialized: now 1786429197963508 us; error 47 us; skew 500 ppm
I20260811 06:19:57.963981 14779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:19:57.968230 14779 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:19:57.974325 14779 webserver.cc:460] Webserver started at http://127.22.121.3:43903/ using document root <none> and password file <none>
I20260811 06:19:57.975004 14779 fs_manager.cc:359] Metadata directory not provided
I20260811 06:19:57.975138 14779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:19:57 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:58.006724 14779 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.011s sys 0.000s
I20260811 06:19:58.010273 14808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:58.012118 14779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:19:58.012363 14779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "cbd6977f2e6e47dab5c29d883d491368"
format_stamp: "Formatted at 2026-08-11 06:19:54 on dist-test-slave-gjrz"
server_key: "f85795cd4a8365ab365c01b0b7c9b3fe"
server_key_iv: "496ba2c594565085a5131910efbd1e6a"
server_key_version: "kuduclusterkey@0"
I20260811 06:19:58.013865 14779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 06:19:58 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for HTTP/127.22.121.20@KRBTEST.COM
I20260811 06:19:58.071302 14779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:19:58.076761 14779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:19:58.077108 14779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:19:58.099687 14779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:19:58.116137 14779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:19:58.116329 14779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:58.116484 14779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:19:58.116605 14779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:19:58.407493 14779 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41733
I20260811 06:19:58.407555 15035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41733 every 8 connection(s)
I20260811 06:19:58.408851 14779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:19:58.414350 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 14779
Aug 11 06:19:58 dist-test-slave-gjrz krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:19:58.430199 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.411257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:42263 (local address 127.22.121.62:40671)
0811 06:19:58.411701 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.411706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.412683 (+ 977us) server_negotiation.cc:407] Connection header received
0811 06:19:58.414384 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.414389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.414487 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.414593 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:58.416388 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.417171 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.418967 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.419455 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.419560 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.423651 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:58.423655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:58.423658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:58.423698 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:58.427047 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.427841 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.427845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.427847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.427958 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.428476 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.428480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.428483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.428777 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:58.428891 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.429571 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.429792 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":218,"threads_started":1}
I20260811 06:19:58.430301 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.415206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.22.121.62:40671)
0811 06:19:58.415750 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.415756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.416004 (+ 248us) server_negotiation.cc:407] Connection header received
0811 06:19:58.416182 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.416186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.416263 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.416352 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:58.417965 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.419154 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.420763 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.421358 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.421466 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.424457 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:58.424464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:58.424468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:58.424528 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:58.427678 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.428379 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.428382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.428384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.428569 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.428903 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.428908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.428911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.429145 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:58.429337 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.429694 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.429975 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":425,"thread_start_us":272,"threads_started":1}
I20260811 06:19:58.431340 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.411836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40671 (local address 127.22.121.3:42263)
0811 06:19:58.412528 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.412571 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.414127 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.414831 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.414841 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.415479 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:19:58.416222 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.416241 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.417324 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.417328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.418825 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.418838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.419857 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.419866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.420144 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.420854 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:19:58.420882 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:19:58.423443 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:19:58.427175 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:58.427182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:58.427194 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:58.427720 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:58.428058 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:19:58.428063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:19:58.428067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:19:58.428366 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:19:58.428895 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:19:58.428901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:19:58.429180 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.429490 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.429992 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":262,"threads_started":1}
I20260811 06:19:58.432660 15036 heartbeater.cc:347] Connected to a master server at 127.22.121.62:40671
I20260811 06:19:58.432962 15036 heartbeater.cc:464] Registering TS with master...
I20260811 06:19:58.433683 15036 heartbeater.cc:511] Master 127.22.121.62:40671 requested a full tablet report, sending...
I20260811 06:19:58.434945 14180 ts_manager.cc:194] Registered new tserver with Master: cbd6977f2e6e47dab5c29d883d491368 (127.22.121.3:41733)
I20260811 06:19:58.435843 14180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:42263
I20260811 06:19:58.446199 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:19:58.461335 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.449123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.22.121.62:40671)
0811 06:19:58.449274 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.449278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.449680 (+ 402us) server_negotiation.cc:407] Connection header received
0811 06:19:58.449798 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.449802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.449863 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.449937 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:58.451356 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.452308 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.453847 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.454489 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.454604 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.457167 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:58.457172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:58.457175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:58.457203 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:58.459452 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.460009 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.460012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.460014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.460136 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:58.460381 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:58.460384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:58.460386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:58.460604 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:58.460774 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.460961 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.461125 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:19:58.465399 14180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:19:58.472568 14180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:19:58.515506 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.486967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33035 (local address 127.0.0.1:56414)
0811 06:19:58.498292 (+ 11325us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.498342 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.498495 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.508240 (+ 9745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.508244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.508271 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:58.509240 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.509255 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.510966 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.510971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.512977 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.512992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.514485 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.514491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.514682 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.514730 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.514860 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.515033 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11098,"mutex_wait_us":84,"thread_start_us":15586,"threads_started":1}
I20260811 06:19:58.515793 15070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.487650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:39053 (local address 127.0.0.1:49144)
0811 06:19:58.504975 (+ 17325us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.504994 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.505236 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.506527 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.506533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.506583 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:58.507832 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.507843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.510295 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.510304 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.513066 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.513081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.514910 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.514916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.515072 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.515101 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.515201 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.515341 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16955,"mutex_wait_us":20,"spinlock_wait_cycles":1792,"thread_start_us":16754,"threads_started":1}
I20260811 06:19:58.515856 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.488391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.22.121.2:39053)
0811 06:19:58.503934 (+ 15543us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.503942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.505101 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 06:19:58.505431 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.505439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.505707 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.505911 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:58.508084 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.510145 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.513253 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.514670 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.514804 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.515097 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.515283 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.515471 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15300,"thread_start_us":15126,"threads_started":1}
I20260811 06:19:58.516479 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.488260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.22.121.1:33035)
0811 06:19:58.507539 (+ 19279us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.507557 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.507585 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:19:58.507680 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.507688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.507958 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.508241 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:58.509573 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.510837 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.513167 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.514284 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.514399 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.514632 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.515984 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.516227 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19114,"thread_start_us":18905,"threads_started":1}
I20260811 06:19:58.516582 15068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.488193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.22.121.3:41733)
0811 06:19:58.505894 (+ 17701us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.505902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.505956 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:19:58.506048 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.506054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.506280 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.506457 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:58.508127 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.509980 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.513430 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.514729 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.514808 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.515013 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.516100 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.516261 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17473,"thread_start_us":18516,"threads_started":1}
I20260811 06:19:58.517270 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.487059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41733 (local address 127.0.0.1:34884)
0811 06:19:58.503019 (+ 15960us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.503038 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.503190 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.506524 (+ 3334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.506533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.506589 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:58.507753 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.507766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.510137 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.510143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.513272 (+ 3129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.513285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.515246 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.515253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.515457 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.515488 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.516863 (+ 1375us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.516981 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15794,"mutex_wait_us":46,"spinlock_wait_cycles":21376,"thread_start_us":10907,"threads_started":1}
I20260811 06:19:58.519685 14774 heartbeater.cc:503] Master 127.22.121.62:40671 was elected leader, sending a full tablet report...
I20260811 06:19:58.520051 14404 tablet_service.cc:1467] Processing CreateTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 (DEFAULT_TABLE table=test-table [id=b53954be21e14b28800df17764b35b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:58.520264 14662 tablet_service.cc:1467] Processing CreateTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 (DEFAULT_TABLE table=test-table [id=b53954be21e14b28800df17764b35b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:58.521605 14404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a610e92d247d4c69acf7c4a3e1f75ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:58.521936 14662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a610e92d247d4c69acf7c4a3e1f75ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:58.534693 14890 tablet_service.cc:1467] Processing CreateTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 (DEFAULT_TABLE table=test-table [id=b53954be21e14b28800df17764b35b3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:19:58.535912 15071 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Bootstrap starting.
I20260811 06:19:58.536489 14890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a610e92d247d4c69acf7c4a3e1f75ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:19:58.540802 15071 tablet_bootstrap.cc:654] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:58.541899 15071 log.cc:824] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:58.543150 15072 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Bootstrap starting.
I20260811 06:19:58.546598 15071 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: No bootstrap required, opened a new log
I20260811 06:19:58.546939 15071 ts_tablet_manager.cc:1397] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260811 06:19:58.548543 15072 tablet_bootstrap.cc:654] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:58.550572 15072 log.cc:824] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:58.554751 15081 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Bootstrap starting.
I20260811 06:19:58.555195 15072 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: No bootstrap required, opened a new log
I20260811 06:19:58.555593 15072 ts_tablet_manager.cc:1397] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260811 06:19:58.559913 15081 tablet_bootstrap.cc:654] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Neither blocks nor log segments found. Creating new log.
I20260811 06:19:58.560997 15071 raft_consensus.cc:348] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.561611 15071 raft_consensus.cc:374] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:58.561776 15071 raft_consensus.cc:729] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc86bb9fd86d4119ac9755c15477519a, State: Initialized, Role: FOLLOWER
I20260811 06:19:58.561828 15081 log.cc:824] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Log is configured to *not* fsync() on all Append() calls
I20260811 06:19:58.562585 15071 consensus_queue.cc:260] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.565589 14516 heartbeater.cc:503] Master 127.22.121.62:40671 was elected leader, sending a full tablet report...
I20260811 06:19:58.565744 15071 ts_tablet_manager.cc:1428] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Time spent starting tablet: real 0.019s user 0.012s sys 0.005s
I20260811 06:19:58.566025 15081 tablet_bootstrap.cc:492] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: No bootstrap required, opened a new log
I20260811 06:19:58.566339 15081 ts_tablet_manager.cc:1397] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260811 06:19:58.568334 15072 raft_consensus.cc:348] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.568929 15072 raft_consensus.cc:374] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:58.569114 15072 raft_consensus.cc:729] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfaaf38158e845aa9dbaad8d40991b2a, State: Initialized, Role: FOLLOWER
W20260811 06:19:58.569160 14519 tablet.cc:2369] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:19:58.569984 15072 consensus_queue.cc:260] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.571852 15072 ts_tablet_manager.cc:1428] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Time spent starting tablet: real 0.016s user 0.002s sys 0.013s
I20260811 06:19:58.576797 15081 raft_consensus.cc:348] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.577142 15081 raft_consensus.cc:374] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:19:58.577268 15081 raft_consensus.cc:729] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd6977f2e6e47dab5c29d883d491368, State: Initialized, Role: FOLLOWER
I20260811 06:19:58.577797 15081 consensus_queue.cc:260] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.579247 15036 heartbeater.cc:503] Master 127.22.121.62:40671 was elected leader, sending a full tablet report...
I20260811 06:19:58.579883 15081 ts_tablet_manager.cc:1428] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260811 06:19:58.660441 15037 tablet.cc:2369] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:19:58.727591 14776 tablet.cc:2369] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:19:58.870409 15087 raft_consensus.cc:482] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:19:58.870669 15087 raft_consensus.cc:504] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.872763 15087 leader_election.cc:288] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbd6977f2e6e47dab5c29d883d491368 (127.22.121.3:41733), dfaaf38158e845aa9dbaad8d40991b2a (127.22.121.2:39053)
I20260811 06:19:58.881711 15068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.873037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:35261 (local address 127.22.121.3:41733)
0811 06:19:58.873331 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.873338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.874842 (+ 1504us) server_negotiation.cc:407] Connection header received
0811 06:19:58.875052 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.875058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.875118 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.875191 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:58.876358 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.877518 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.879893 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.880781 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.880942 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.880994 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.881346 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.881444 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 06:19:58.882182 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.873613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41733 (local address 127.22.121.1:35261)
0811 06:19:58.874717 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.874751 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.874916 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.875289 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.875295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.875320 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:58.876215 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.876228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.877677 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.877681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.879721 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.879734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.881016 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.881023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.881196 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.881232 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.881337 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.881459 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":18,"thread_start_us":670,"threads_started":1}
I20260811 06:19:58.882611 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.873139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:45727 (local address 127.22.121.2:39053)
0811 06:19:58.873340 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.873345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.875486 (+ 2141us) server_negotiation.cc:407] Connection header received
0811 06:19:58.875736 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.875740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.875799 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.875876 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:19:58.876918 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.877749 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.879963 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.881349 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.881527 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.881598 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.882256 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.882379 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:19:58.882792 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.873648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:39053 (local address 127.22.121.1:45727)
0811 06:19:58.875385 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0811 06:19:58.875400 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:19:58.875567 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:19:58.875911 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:19:58.875915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:19:58.875953 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:19:58.876711 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.876722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.877902 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.877907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.879826 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:19:58.879839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.881806 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.881816 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:19:58.882044 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.882079 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:19:58.882190 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:19:58.882358 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537,"mutex_wait_us":1,"thread_start_us":1214,"threads_started":1}
I20260811 06:19:58.883509 14956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a610e92d247d4c69acf7c4a3e1f75ce4" candidate_uuid: "dc86bb9fd86d4119ac9755c15477519a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd6977f2e6e47dab5c29d883d491368" is_pre_election: true
I20260811 06:19:58.883818 14695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a610e92d247d4c69acf7c4a3e1f75ce4" candidate_uuid: "dc86bb9fd86d4119ac9755c15477519a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" is_pre_election: true
I20260811 06:19:58.884253 14956 raft_consensus.cc:2454] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc86bb9fd86d4119ac9755c15477519a in term 0.
I20260811 06:19:58.884545 14695 raft_consensus.cc:2454] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc86bb9fd86d4119ac9755c15477519a in term 0.
I20260811 06:19:58.884966 14327 leader_election.cc:302] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd6977f2e6e47dab5c29d883d491368, dc86bb9fd86d4119ac9755c15477519a; no voters:
I20260811 06:19:58.885479 15087 raft_consensus.cc:2781] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:19:58.885635 15087 raft_consensus.cc:482] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:19:58.885735 15087 raft_consensus.cc:3037] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:58.889174 15087 raft_consensus.cc:504] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.889829 15087 leader_election.cc:288] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [CANDIDATE]: Term 1 election: Requested vote from peers cbd6977f2e6e47dab5c29d883d491368 (127.22.121.3:41733), dfaaf38158e845aa9dbaad8d40991b2a (127.22.121.2:39053)
I20260811 06:19:58.890231 14956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a610e92d247d4c69acf7c4a3e1f75ce4" candidate_uuid: "dc86bb9fd86d4119ac9755c15477519a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd6977f2e6e47dab5c29d883d491368"
I20260811 06:19:58.890337 14695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a610e92d247d4c69acf7c4a3e1f75ce4" candidate_uuid: "dc86bb9fd86d4119ac9755c15477519a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfaaf38158e845aa9dbaad8d40991b2a"
I20260811 06:19:58.890450 14956 raft_consensus.cc:3037] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:58.890578 14695 raft_consensus.cc:3037] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:19:58.893539 14956 raft_consensus.cc:2454] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc86bb9fd86d4119ac9755c15477519a in term 1.
I20260811 06:19:58.893738 14695 raft_consensus.cc:2454] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc86bb9fd86d4119ac9755c15477519a in term 1.
I20260811 06:19:58.893970 14327 leader_election.cc:302] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd6977f2e6e47dab5c29d883d491368, dc86bb9fd86d4119ac9755c15477519a; no voters:
I20260811 06:19:58.894234 15087 raft_consensus.cc:2781] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:19:58.894656 15087 raft_consensus.cc:686] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 1 LEADER]: Becoming Leader. State: Replica: dc86bb9fd86d4119ac9755c15477519a, State: Running, Role: LEADER
I20260811 06:19:58.895152 15087 consensus_queue.cc:237] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } }
I20260811 06:19:58.899696 14180 catalog_manager.cc:5495] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a reported cstate change: term changed from 0 to 1, leader changed from <none> to dc86bb9fd86d4119ac9755c15477519a (127.22.121.1). New cstate: current_term: 1 leader_uuid: "dc86bb9fd86d4119ac9755c15477519a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc86bb9fd86d4119ac9755c15477519a" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260811 06:19:58.961530 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:58.956642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56426 (local address 127.22.121.1:33035)
0811 06:19:58.956838 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:58.956843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:58.956860 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:19:58.956996 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:58.957001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:58.957084 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:58.957208 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:19:58.957528 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.958243 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.959633 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:58.960179 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:58.960271 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:58.960652 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:19:58.961105 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:19:58.961208 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:58.961247 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 06:19:58.961307 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:19:58.970417 14956 raft_consensus.cc:1261] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 1 FOLLOWER]: Refusing update from remote peer dc86bb9fd86d4119ac9755c15477519a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:19:58.970459 14695 raft_consensus.cc:1261] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 1 FOLLOWER]: Refusing update from remote peer dc86bb9fd86d4119ac9755c15477519a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:19:58.971414 15093 consensus_queue.cc:1035] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd6977f2e6e47dab5c29d883d491368" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:19:58.971946 15087 consensus_queue.cc:1035] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfaaf38158e845aa9dbaad8d40991b2a" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:19:58.984928 15097 mvcc.cc:205] Tried to move back new op lower bound from 7317213998972973056 to 7317213998678167552. Current Snapshot: MvccSnapshot[applied={T|T < 7317213998972973056}]
I20260811 06:19:58.998433 14181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:19:58.998797 14181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:19:59.002836 14181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d14ddf4b2614e52a52ee1873ec825e8: Sending DeleteTablet for 3 replicas of tablet a610e92d247d4c69acf7c4a3e1f75ce4
I20260811 06:19:59.003978 14404 tablet_service.cc:1514] Processing DeleteTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:56414
I20260811 06:19:59.003981 14890 tablet_service.cc:1514] Processing DeleteTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:34884
I20260811 06:19:59.004081 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet a610e92d247d4c69acf7c4a3e1f75ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:19:58 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:49144
I20260811 06:19:59.005328 15116 tablet_replica.cc:330] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: stopping tablet replica
I20260811 06:19:59.005716 15116 raft_consensus.cc:2229] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:19:59.006165 15116 raft_consensus.cc:2258] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:59.006848 15119 tablet_replica.cc:330] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: stopping tablet replica
I20260811 06:19:59.007218 15119 raft_consensus.cc:2229] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:19:59.007879 15119 raft_consensus.cc:2258] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:59.008683 15120 tablet_replica.cc:330] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: stopping tablet replica
I20260811 06:19:59.009065 15120 raft_consensus.cc:2229] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:19:59.009508 15120 raft_consensus.cc:2258] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:19:59.010118 15119 ts_tablet_manager.cc:1905] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:19:59.011579 15120 ts_tablet_manager.cc:1905] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:19:59.012823 15116 ts_tablet_manager.cc:1905] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:19:59.015540 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 14246
I20260811 06:19:59.019704 15120 ts_tablet_manager.cc:1918] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:19:59.020006 15120 log.cc:1196] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/a610e92d247d4c69acf7c4a3e1f75ce4
I20260811 06:19:59.020753 15120 ts_tablet_manager.cc:1939] T a610e92d247d4c69acf7c4a3e1f75ce4 P dfaaf38158e845aa9dbaad8d40991b2a: Deleting consensus metadata
I20260811 06:19:59.022737 14154 catalog_manager.cc:4841] TS dfaaf38158e845aa9dbaad8d40991b2a (127.22.121.2:39053): tablet a610e92d247d4c69acf7c4a3e1f75ce4 (table test-table [id=b53954be21e14b28800df17764b35b3c]) successfully deleted
I20260811 06:19:59.023404 15119 ts_tablet_manager.cc:1918] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:19:59.023504 15116 ts_tablet_manager.cc:1918] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:19:59.023654 15119 log.cc:1196] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/a610e92d247d4c69acf7c4a3e1f75ce4
I20260811 06:19:59.023751 15116 log.cc:1196] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/a610e92d247d4c69acf7c4a3e1f75ce4
I20260811 06:19:59.024290 15116 ts_tablet_manager.cc:1939] T a610e92d247d4c69acf7c4a3e1f75ce4 P cbd6977f2e6e47dab5c29d883d491368: Deleting consensus metadata
I20260811 06:19:59.024437 15119 ts_tablet_manager.cc:1939] T a610e92d247d4c69acf7c4a3e1f75ce4 P dc86bb9fd86d4119ac9755c15477519a: Deleting consensus metadata
I20260811 06:19:59.025946 14156 catalog_manager.cc:4841] TS cbd6977f2e6e47dab5c29d883d491368 (127.22.121.3:41733): tablet a610e92d247d4c69acf7c4a3e1f75ce4 (table test-table [id=b53954be21e14b28800df17764b35b3c]) successfully deleted
I20260811 06:19:59.026309 14148 catalog_manager.cc:4841] TS dc86bb9fd86d4119ac9755c15477519a (127.22.121.1:33035): tablet a610e92d247d4c69acf7c4a3e1f75ce4 (table test-table [id=b53954be21e14b28800df17764b35b3c]) successfully deleted
I20260811 06:19:59.040303 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:59.016859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.22.121.1:33035)
0811 06:19:59.017077 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:59.017084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:59.026381 (+ 9297us) server_negotiation.cc:407] Connection header received
0811 06:19:59.026556 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:59.026560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:59.026617 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:59.026698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:59.028383 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.029201 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.030973 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.031494 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.031592 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:59.034624 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:59.034634 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:59.034646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:59.034684 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:59.037524 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.038325 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.038331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.038333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.038481 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.038865 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.038868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.038870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.039200 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:59.039475 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:59.039839 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:19:59.040029 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 06:19:59.040992 14509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:59.140026 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 14246
I20260811 06:19:59.163254 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 14522
I20260811 06:19:59.175254 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:59.164006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.22.121.2:39053)
0811 06:19:59.164198 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:59.164202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:59.164220 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:19:59.164355 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:59.164358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:59.164421 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:59.164510 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:59.165618 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.166408 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.167739 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.168133 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.168222 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:59.170402 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:59.170415 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:59.170430 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:59.170460 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:59.172942 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.173587 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.173594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.173596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.173736 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.174027 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.174030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.174032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.174358 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:59.174573 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:59.174761 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 06:19:59.174918 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:19:59.175925 14768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:59.264861 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 14522
I20260811 06:19:59.291992 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 14779
I20260811 06:19:59.304183 15068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:19:59.292776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.22.121.3:41733)
0811 06:19:59.292938 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:19:59.292942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:19:59.292961 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:19:59.293074 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:19:59.293078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:19:59.293144 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:19:59.293226 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:19:59.294445 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.295161 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.296418 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:19:59.296818 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:19:59.296907 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:19:59.299215 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:19:59.299230 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:19:59.299249 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:19:59.299302 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:19:59.301969 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.302561 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.302568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.302571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.302703 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:19:59.302952 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:19:59.302955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:19:59.302956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:19:59.303315 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:19:59.303537 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:19:59.303759 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 06:19:59.303912 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:19:59.304826 15028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:59.390141 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 14779
I20260811 06:19:59.415786 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 14119
I20260811 06:19:59.416898 14234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:19:59.526567 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 14119
I20260811 06:19:59.565656 23012 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:19:59.910303 23012 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:19:59.910487 23012 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:20:00.254081 23012 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:20:00.254 UTC [12488] LOG: received smart shutdown request
2026-08-11 06:20:00.257 UTC [12488] LOG: background worker "logical replication launcher" (PID 12506) exited with exit code 1
2026-08-11 06:20:00.257 UTC [12493] LOG: shutting down
2026-08-11 06:20:00.381 UTC [12488] LOG: database system is shut down
2026-08-11T06:20:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51202 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:20:00 dist-test-slave-gjrz krb5kdc[15145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:20:00 dist-test-slave-gjrz krb5kdc[15145](info): set up 2 sockets
Aug 11 06:20:00 dist-test-slave-gjrz krb5kdc[15145](info): commencing operation
krb5kdc: starting...
W20260811 06:20:01.718852 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:20:01 dist-test-slave-gjrz krb5kdc[15145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:20:01.926324 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40379
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:02.240097 15163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:02.240350 15163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:02.240490 15163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:02.247351 15163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:02.247452 15163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:02.247526 15163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:02.247604 15163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:02.247664 15163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:02.256850 15163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:02.257953 15163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:02.259171 15163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:02.269974 15171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:02.269919 15173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:02.270021 15176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:02.271027 15163 server_base.cc:1034] running on GCE node
I20260811 06:20:02.271766 15163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:02.273072 15163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:02.274268 15163 hybrid_clock.cc:648] HybridClock initialized: now 1786429202274230 us; error 25 us; skew 500 ppm
I20260811 06:20:02.274587 15163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:02 dist-test-slave-gjrz krb5kdc[15145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:02.277509 15163 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:02.278787 15163 webserver.cc:460] Webserver started at http://127.22.121.62:41691/ using document root <none> and password file <none>
I20260811 06:20:02.279433 15163 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:02.279562 15163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:02.279837 15163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:02.282653 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "a2fee4cfcaa145918e23119294b6083d"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "ac376a240ff55f17f4b3e3331ac7ddbb"
server_key_iv: "7f249ff6e532ce1f6cfb1b740771c576"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.283497 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "a2fee4cfcaa145918e23119294b6083d"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "ac376a240ff55f17f4b3e3331ac7ddbb"
server_key_iv: "7f249ff6e532ce1f6cfb1b740771c576"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.288262 15163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 06:20:02.291512 15185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:02.292723 15163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:02.292868 15163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a2fee4cfcaa145918e23119294b6083d"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "ac376a240ff55f17f4b3e3331ac7ddbb"
server_key_iv: "7f249ff6e532ce1f6cfb1b740771c576"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.293035 15163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:02.313526 15163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:02.317090 15163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:02.317327 15163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:02.334352 15163 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:40379
I20260811 06:20:02.334405 15274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:40379 every 8 connection(s)
I20260811 06:20:02.335552 15163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:02.338835 15279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:02.343520 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 15163
I20260811 06:20:02.343713 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:20:02.344336 23012 external_mini_cluster.cc:1428] Setting key 861d400e25df753dde99c91930edf791
I20260811 06:20:02.345090 15279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:20:02 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.61@KRBTEST.COM" created.
I20260811 06:20:02.370234 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.347962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40379 (local address 127.0.0.1:35412)
0811 06:20:02.348967 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.349006 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:02.350628 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:02.351381 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:02.351388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:02.352057 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:02.352364 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.352381 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.353852 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.353856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.354876 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.354888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.355781 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.355788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.355946 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.361023 (+ 5077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:02.361044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:02.363236 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:02.366371 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.366383 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.366394 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.366763 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.367140 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.367145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.367149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.367358 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.368056 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:02.368064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:02.368276 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 06:20:02.368479 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 06:20:02.369044 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":142,"thread_start_us":634,"threads_started":1}
I20260811 06:20:02.370253 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.347957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.22.121.62:40379)
0811 06:20:02.348681 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:02.348696 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:02.349116 (+ 420us) server_negotiation.cc:407] Connection header received
0811 06:20:02.350958 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:02.350964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:02.351172 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:02.351366 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:02.352554 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.353717 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.355004 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.355568 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.355729 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.363452 (+ 7723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:02.363460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:02.363472 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:02.363504 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:02.366254 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.366883 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.366891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.366893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.367035 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.367458 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.367463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.367466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.367917 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:02.368103 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:02.368640 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 06:20:02.368886 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":334,"threads_started":1}
I20260811 06:20:02.373080 15279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:02.374046 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.373625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52566)
0811 06:20:02.373778 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.373892 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 06:20:02.374266 15197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
I20260811 06:20:02.375705 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.375306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52580)
0811 06:20:02.375463 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.375544 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:20:02.375953 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:42435: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
I20260811 06:20:02.392072 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:42435
--webserver_interface=127.22.121.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:02.425352 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } attempt: 1
I20260811 06:20:02.426508 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.425997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52596)
0811 06:20:02.426202 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.426342 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 06:20:02.427521 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.427103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52606)
0811 06:20:02.427248 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.427360 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:20:02.427814 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:42435: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
I20260811 06:20:02.496097 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } attempt: 2
I20260811 06:20:02.497326 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.496778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52618)
0811 06:20:02.497030 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.497143 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 06:20:02.498369 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.497962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52634)
0811 06:20:02.498130 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.498218 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:20:02.498593 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:42435: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
I20260811 06:20:02.589844 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } attempt: 3
I20260811 06:20:02.591061 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.590546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52642)
0811 06:20:02.590768 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.590894 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:20:02.592154 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.591710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52654)
0811 06:20:02.591896 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.591986 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:20:02.592334 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:42435: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
W20260811 06:20:02.702829 15290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:02.703081 15290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:02.703166 15290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:02.709936 15290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:02.710023 15290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:02.710075 15290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:02.710130 15290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:02.710172 15290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:02.719168 15290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.121.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:02.720252 15290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:02.721386 15290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:02.732192 15298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:02.732125 15299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:02.732228 15302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:02.732805 15290 server_base.cc:1034] running on GCE node
I20260811 06:20:02.733503 15290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:02.734740 15290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:02.735545 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } attempt: 4
I20260811 06:20:02.735948 15290 hybrid_clock.cc:648] HybridClock initialized: now 1786429202735896 us; error 31 us; skew 500 ppm
I20260811 06:20:02.736330 15290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:20:02.736615 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.736115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52662)
0811 06:20:02.736388 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.736480 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 06:20:02.737641 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.737264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52676)
0811 06:20:02.737420 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.737525 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:20:02.737818 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:42435: Network error: Client connection negotiation failed: client connection to 127.22.121.61:42435: connect: Connection refused (error 111)
Aug 11 06:20:02 dist-test-slave-gjrz krb5kdc[15145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:02.739590 15290 init.cc:377] Logged in from keytab as kudu/127.22.121.61@KRBTEST.COM (short username kudu)
I20260811 06:20:02.740872 15290 webserver.cc:460] Webserver started at http://127.22.121.61:39051/ using document root <none> and password file <none>
I20260811 06:20:02.741492 15290 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:02.741616 15290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:02.741881 15290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:02.744658 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data/instance:
uuid: "f80e91f3041146ffaf8276642e0fc40b"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "2a4f4148ae5dd74fe3e6ae990c5d6b49"
server_key_iv: "2c7db470811aa92789c12bf490b05dc1"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.745395 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal/instance:
uuid: "f80e91f3041146ffaf8276642e0fc40b"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "2a4f4148ae5dd74fe3e6ae990c5d6b49"
server_key_iv: "2c7db470811aa92789c12bf490b05dc1"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.750051 15290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:20:02.753152 15317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:02.754304 15290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:20:02.754447 15290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "f80e91f3041146ffaf8276642e0fc40b"
format_stamp: "Formatted at 2026-08-11 06:20:02 on dist-test-slave-gjrz"
server_key: "2a4f4148ae5dd74fe3e6ae990c5d6b49"
server_key_iv: "2c7db470811aa92789c12bf490b05dc1"
server_key_version: "encryptionkey@0"
I20260811 06:20:02.754607 15290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:02.777953 15290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:02.781848 15290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:02.782117 15290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:02.798066 15290 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.61:42435
I20260811 06:20:02.798089 15401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.61:42435 every 8 connection(s)
I20260811 06:20:02.799239 15290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
I20260811 06:20:02.800077 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 15290
I20260811 06:20:02.800262 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-1/wal/instance
I20260811 06:20:02.800657 23012 external_mini_cluster.cc:1428] Setting key 00656b628477fd65c9cc84b326774163
I20260811 06:20:02.802729 15402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:02.808836 15402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:20:02 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.60@KRBTEST.COM" created.
I20260811 06:20:02.828903 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.811761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.22.121.62:40379)
0811 06:20:02.811950 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:02.811956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:02.813118 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 06:20:02.814883 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:02.814888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:02.814966 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:02.815082 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:02.816444 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.817547 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.819100 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.819758 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.819892 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.823297 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:02.823304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:02.823309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:02.823349 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:02.826116 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.826776 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.826782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.826786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.826905 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.827297 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.827307 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.827311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.827590 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:02.827720 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:02.828369 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 06:20:02.828606 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:20:02.829739 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.812221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40379 (local address 127.0.0.1:35418)
0811 06:20:02.812953 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.813003 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:02.814641 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:02.815237 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:02.815244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:02.815766 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:02.816269 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.816285 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.817731 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.817735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.818937 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.818950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.819952 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.819957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.820146 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.820779 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:02.820800 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:02.823064 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:02.826249 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.826257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.826272 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.826668 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.827013 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.827018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.827022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.827180 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.827708 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:02.827715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:02.828035 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 06:20:02.828259 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 06:20:02.828701 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":311,"threads_started":1}
I20260811 06:20:02.831419 15402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:20:02 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:20:02.844022 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.831991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52686)
0811 06:20:02.832107 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.832120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:02.832189 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:02.833078 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:02.833082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:02.833441 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:02.833561 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.833568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.834812 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.834816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.835667 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:02.835678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.836417 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.836422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:02.836581 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.837064 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:02.837079 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:02.839025 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:02.841779 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.841784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.841788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.842131 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.842449 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:02.842454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:02.842458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:02.842586 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:02.843115 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:02.843119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:02.843246 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 06:20:02.843404 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:20:02.843641 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 06:20:02.844111 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.832228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.22.121.61:42435)
0811 06:20:02.832712 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:02.832718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:02.832742 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:20:02.832779 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:02.832784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:02.832939 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:02.833069 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:02.833699 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.834691 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.835784 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:02.836275 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:02.836379 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:02.839197 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:02.839207 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:02.839218 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:02.839248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:02.841671 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.842241 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.842248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.842250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.842355 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:02.842684 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:02.842689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:02.842693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:02.843000 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:02.843183 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:02.843580 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 06:20:02.843775 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260811 06:20:02.846412 15402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:02.847536 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.847109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36246)
0811 06:20:02.847285 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.847401 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 06:20:02.847796 15319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:02.849159 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.848870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36252)
0811 06:20:02.848995 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.849068 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 06:20:02.849354 15402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
I20260811 06:20:02.852162 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:35897
--webserver_interface=127.22.121.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:02.898638 15402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 1
I20260811 06:20:02.899778 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.899346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36260)
0811 06:20:02.899526 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.899630 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 06:20:02.900714 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.900369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36270)
0811 06:20:02.900522 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.900596 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:20:02.900921 15402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:02.969126 15402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 2
I20260811 06:20:02.970216 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.969779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36278)
0811 06:20:02.969965 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.970055 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:20:02.971134 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:02.970799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36280)
0811 06:20:02.970938 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:02.971009 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 06:20:02.971338 15402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:03.037041 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } attempt: 5
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:20:03.050105 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.037912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.22.121.61:42435)
0811 06:20:03.038123 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.038129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.038149 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:20:03.038207 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.038211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.038288 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.038373 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.039246 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.039992 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.041238 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.041701 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.041814 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.045018 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.045022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.045026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.045065 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.047663 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.048296 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.048301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.048304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.048419 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.048805 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.048809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.048811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.049080 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.049213 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.049586 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.049791 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:20:03.050112 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.037708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52700)
0811 06:20:03.037898 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.037918 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.038043 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.038453 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.038458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.038852 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.039068 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.039080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.040148 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.040152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.041095 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.041110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.041845 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.041852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.042062 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.042676 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.042693 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.044837 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.047807 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.047813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.047818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.048182 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.048549 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.048554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.048558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.048701 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.049198 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.049202 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.049356 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.049477 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.049745 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:20:03.051251 15279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:03.052376 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.052016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36296)
0811 06:20:03.052174 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.052244 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:20:03.053325 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.053018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36312)
0811 06:20:03.053151 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.053229 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:20:03.053537 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:03.062528 15402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 3
I20260811 06:20:03.063629 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.063092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36322)
0811 06:20:03.063343 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.063455 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:20:03.064689 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.064230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36324)
0811 06:20:03.064452 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.064546 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260811 06:20:03.064889 15402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:03.104758 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 1
I20260811 06:20:03.105854 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.105402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36340)
0811 06:20:03.105597 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.105700 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:20:03.106788 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.106488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36356)
0811 06:20:03.106612 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.106682 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:20:03.106988 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
W20260811 06:20:03.165654 15410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:03.165904 15410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:03.165987 15410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:03.172821 15410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:03.172914 15410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:03.172966 15410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:03.173017 15410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:03.173059 15410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:03.175184 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 2
I20260811 06:20:03.176375 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.175879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36360)
0811 06:20:03.176115 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.176212 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 06:20:03.177367 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.177017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36370)
0811 06:20:03.177166 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.177254 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:20:03.177573 15279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:03.182595 15410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:40379,127.22.121.61:42435,127.22.121.60:35897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:35897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.121.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:03.183794 15410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:03.184988 15410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:03.195914 15419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:03.195981 15420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:03.196003 15423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:03.196710 15410 server_base.cc:1034] running on GCE node
I20260811 06:20:03.197412 15410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:03.198660 15410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:03.199851 15410 hybrid_clock.cc:648] HybridClock initialized: now 1786429203199797 us; error 47 us; skew 500 ppm
I20260811 06:20:03.200186 15410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:03.203042 15410 init.cc:377] Logged in from keytab as kudu/127.22.121.60@KRBTEST.COM (short username kudu)
I20260811 06:20:03.204434 15410 webserver.cc:460] Webserver started at http://127.22.121.60:39017/ using document root <none> and password file <none>
I20260811 06:20:03.205054 15410 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:03.205178 15410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:03.205447 15410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:03.208082 15402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 4
I20260811 06:20:03.208284 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data/instance:
uuid: "e4029c6166f34b68a7072595056a6dc6"
format_stamp: "Formatted at 2026-08-11 06:20:03 on dist-test-slave-gjrz"
server_key: "54e85ac5e624d727e45f3485b2820c9e"
server_key_iv: "10c3c74f67c7ea910dc79b8e4c179969"
server_key_version: "encryptionkey@0"
I20260811 06:20:03.209002 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.208563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36372)
0811 06:20:03.208762 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.208837 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:20:03.209057 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal/instance:
uuid: "e4029c6166f34b68a7072595056a6dc6"
format_stamp: "Formatted at 2026-08-11 06:20:03 on dist-test-slave-gjrz"
server_key: "54e85ac5e624d727e45f3485b2820c9e"
server_key_iv: "10c3c74f67c7ea910dc79b8e4c179969"
server_key_version: "encryptionkey@0"
I20260811 06:20:03.209872 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.209567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36388)
0811 06:20:03.209705 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.209773 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:20:03.210059 15402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:35897: Network error: Client connection negotiation failed: client connection to 127.22.121.60:35897: connect: Connection refused (error 111)
I20260811 06:20:03.213946 15410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260811 06:20:03.217136 15434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:03.218302 15410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:20:03.218449 15410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "e4029c6166f34b68a7072595056a6dc6"
format_stamp: "Formatted at 2026-08-11 06:20:03 on dist-test-slave-gjrz"
server_key: "54e85ac5e624d727e45f3485b2820c9e"
server_key_iv: "10c3c74f67c7ea910dc79b8e4c179969"
server_key_version: "encryptionkey@0"
I20260811 06:20:03.218617 15410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:03.245046 15410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:03.248597 15410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:03.248811 15410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:03.265255 15410 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.60:35897
I20260811 06:20:03.265286 15534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.60:35897 every 8 connection(s)
I20260811 06:20:03.266426 15410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
I20260811 06:20:03.269731 15535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:03.271842 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 15410
I20260811 06:20:03.272032 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429000386903-23012-0/minicluster-data/master-2/wal/instance
I20260811 06:20:03.272563 23012 external_mini_cluster.cc:1428] Setting key 7ec270efcc0efd0dce751eaf98a826b4
I20260811 06:20:03.277207 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:03.283768 15279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 3
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:20:03.295105 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.279991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.22.121.62:40379)
0811 06:20:03.280159 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.280163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.280833 (+ 670us) server_negotiation.cc:407] Connection header received
0811 06:20:03.280998 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.281004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.281078 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.281157 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.282192 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.282868 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.284431 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.285096 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.285186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.288945 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.288950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.288953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.288982 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.292193 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.292969 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.292975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.292978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.293109 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.293510 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.293515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.293518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.293838 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.293986 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.294539 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.294743 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:20:03.296284 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.280051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:40379 (local address 127.0.0.1:35434)
0811 06:20:03.280697 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.280736 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.280875 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.281233 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.281239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.281778 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.282049 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.282060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.283018 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.283022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.284281 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.284296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.285519 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.285525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.285651 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.286178 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.286206 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.288748 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.292299 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.292310 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.292318 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.292861 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.293209 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.293214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.293218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.293390 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.293949 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.293956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.294169 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.294438 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.294936 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":334,"threads_started":1}
I20260811 06:20:03.296314 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.274364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.22.121.60:35897)
0811 06:20:03.275135 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.275148 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.275193 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:20:03.276812 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.276834 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.277199 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.277506 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.279313 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.281155 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.282583 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.283211 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.283412 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.287835 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.287845 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.287858 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.287894 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.291345 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.292177 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.292185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.292193 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.292390 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.292857 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.292862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.292865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.293532 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.293855 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.294872 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.295137 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":351,"threads_started":1}
I20260811 06:20:03.298305 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:03.299450 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.284271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36412)
0811 06:20:03.284496 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.284513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.284597 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.285357 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.285362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.285820 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.286018 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.286030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.287654 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.287660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.288998 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.289016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.289976 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.289983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.290166 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.290736 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.290760 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.293272 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.297133 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.297138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.297142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.297588 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.297972 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.297977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.297981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.298097 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.298591 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.298597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.298749 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.298874 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.299104 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 06:20:03.300639 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.284336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.22.121.60:35897)
0811 06:20:03.285036 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.285041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.285075 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:20:03.285116 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.285120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.285188 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.285286 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.286199 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.287492 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.289153 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.289762 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.289933 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.293486 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.293492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.293496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.293527 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.296977 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.297722 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.297727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.297729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.297871 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.298197 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.298202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.298206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.298481 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.298633 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.300197 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.300373 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":437,"threads_started":1}
I20260811 06:20:03.305622 15279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: Bootstrap starting.
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:03.309571 15279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:03.310871 15279 log.cc:824] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:03.312496 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.298885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:42435 (local address 127.0.0.1:52710)
0811 06:20:03.298996 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.299013 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.299098 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.299988 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.299992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.300470 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.300665 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.300677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.302010 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.302015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.303286 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.303300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.304189 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.304196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.304429 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.304949 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.304969 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.307045 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.310042 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.310047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.310051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.310472 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.310897 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.310902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.310906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.311086 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.311675 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.311679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.311824 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.311936 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.312180 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 06:20:03.312821 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.299211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.22.121.61:42435)
0811 06:20:03.299418 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.299425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.299444 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:20:03.299680 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.299686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.299764 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.299823 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.300841 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.301820 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.303406 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.304031 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.304162 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.307231 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.307235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.307238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.307325 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.309920 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.310598 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.310604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.310607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.310772 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.311206 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.311211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.311214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.311573 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.311721 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.312048 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.312247 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:20:03.313592 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:20:03.315011 15279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: No bootstrap required, opened a new log
I20260811 06:20:03.317482 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.299661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.22.121.62:40379)
0811 06:20:03.299827 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.299830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.299842 (+ 12us) server_negotiation.cc:407] Connection header received
0811 06:20:03.299874 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.299877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.299920 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.299985 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.301874 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.303110 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.304744 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.305364 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.305489 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.310321 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.310327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.310331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.310367 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.313693 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.314783 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.314789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.314792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.314955 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.315358 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.315363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.315367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.315646 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.315911 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.316921 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.317144 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:20:03.324612 15279 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.324977 15279 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:03.325103 15279 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fee4cfcaa145918e23119294b6083d, State: Initialized, Role: FOLLOWER
I20260811 06:20:03.325737 15279 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.326891 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.327383 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.327770 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.314405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36416)
0811 06:20:03.314539 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.314555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.314647 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.314951 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.314956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.315411 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.315595 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.315608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.316840 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.316845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.318102 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.318116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.319021 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.319027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.319202 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.319725 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.319745 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.321648 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.325235 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.325240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.325243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.325690 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.326090 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.326094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.326097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.326251 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.326895 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.326900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.327041 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.327167 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.327471 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:20:03.328243 15279 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:03.328755 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.314320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.22.121.60:35897)
0811 06:20:03.314462 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.314467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.314615 (+ 148us) server_negotiation.cc:407] Connection header received
0811 06:20:03.314727 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.314732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.314807 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.314896 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.315766 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.316706 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.318251 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.318860 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.318986 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.321839 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.321846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.321850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.321896 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.325098 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.325816 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.325821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.325823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.325963 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.326395 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.326400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.326404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.326768 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.328160 (+ 1392us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.328270 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.328462 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 06:20:03.333089 15535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: Bootstrap starting.
I20260811 06:20:03.336678 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.319392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.22.121.61:42435)
0811 06:20:03.319561 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.319567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.319583 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:20:03.319728 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.319733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.319797 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.319893 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.321376 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.322275 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.323776 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.324180 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.324247 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.329300 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.329307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.329313 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.329353 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.332832 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.333596 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.333602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.333606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.333775 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.334241 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.334247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.334250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.334605 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.335952 (+ 1347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.336090 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.336320 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:03.336827 15535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:03.338083 15535 log.cc:824] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:03.343092 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:20:03.352330 15535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: No bootstrap required, opened a new log
I20260811 06:20:03.362557 15535 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.363034 15535 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:03.363224 15535 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4029c6166f34b68a7072595056a6dc6, State: Initialized, Role: FOLLOWER
I20260811 06:20:03.364043 15535 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.371052 15575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.373324 15575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.373200 15535 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:03.374742 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.355139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.22.121.60:35897)
0811 06:20:03.355320 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.355326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.356940 (+ 1614us) server_negotiation.cc:407] Connection header received
0811 06:20:03.357119 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.357125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.357215 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.357313 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.359820 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.361025 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.363218 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.363963 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.364061 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.367786 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.367792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.367796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.367830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.371249 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.372275 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.372280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.372284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.372438 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.372956 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.372961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.372964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.373283 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.373536 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.374127 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.374339 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:20:03.381507 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.354577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.22.121.62:40379)
0811 06:20:03.354731 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.354737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.361786 (+ 7049us) server_negotiation.cc:407] Connection header received
0811 06:20:03.363005 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.363010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.363086 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.363157 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.366303 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.367473 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.369675 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.370381 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.370473 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.373974 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.373981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.373986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.374034 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.377842 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.378971 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.378977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.378981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.379188 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.379692 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.379698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.379702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.380053 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.380287 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.380886 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.381130 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:03.385480 15582 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:20:03.387163 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.362228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.22.121.61:42435)
0811 06:20:03.362375 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.362380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.366503 (+ 4123us) server_negotiation.cc:407] Connection header received
0811 06:20:03.367534 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.367539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.367609 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.367671 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.371458 (+ 3787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.372454 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.376270 (+ 3816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.376866 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.376957 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.380242 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.380249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.380253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.380294 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.383940 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.385005 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.385012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.385016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.385177 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.385633 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.385638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.385641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.385999 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.386208 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.386671 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.386876 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:03.389678 15163 master_runner.cc:186] Error getting master registration for 127.22.121.61:42435: OK, instance_id { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" instance_seqno: 1786429202793435 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:20:03.399312 15163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:20:03.402755 15603 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:20:03.402880 15603 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:20:03.401028 15410 master_runner.cc:186] Error getting master registration for 127.22.121.61:42435: OK, instance_id { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" instance_seqno: 1786429202793435 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:20:03.408831 15410 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:20:03.500609 15445 tablet.cc:2369] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:20:03.509240 15402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } attempt: 5
Aug 11 06:20:03 dist-test-slave-gjrz krb5kdc[15145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:20:03.522332 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.509757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:35897 (local address 127.0.0.1:36444)
0811 06:20:03.509928 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:03.509944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:03.510027 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:03.510323 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:03.510328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:03.510735 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:03.510887 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.510895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.511990 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.511994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.512871 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:03.512882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.513601 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.513607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:03.513799 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.514365 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:03.514381 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:03.516710 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:03.519604 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.519610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.519614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.520118 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.520452 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:03.520457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:03.520461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:03.520631 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:03.521358 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:03.521363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:03.521522 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 06:20:03.521675 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 06:20:03.521962 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:20:03.522557 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:03.509803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.22.121.60:35897)
0811 06:20:03.509971 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:03.509977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:03.510001 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:20:03.510151 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:03.510156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:03.510210 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:03.510286 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:03.511023 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.511870 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.513006 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:03.513474 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:03.513588 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:03.516870 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:03.516875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:03.516878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:03.516906 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:03.519458 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.520219 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.520223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.520225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.520347 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:03.520769 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:03.520774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:03.520778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:03.521156 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:03.521294 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:03.521999 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 06:20:03.522224 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:20:03.526675 15402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: Bootstrap starting.
I20260811 06:20:03.529925 15402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:03.530889 15402 log.cc:824] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:03.533319 15402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: No bootstrap required, opened a new log
I20260811 06:20:03.541033 15402 raft_consensus.cc:348] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.541313 15402 raft_consensus.cc:374] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:03.541421 15402 raft_consensus.cc:729] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80e91f3041146ffaf8276642e0fc40b, State: Initialized, Role: FOLLOWER
I20260811 06:20:03.541954 15402 consensus_queue.cc:260] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.542752 15607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.543115 15607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.543854 15402 sys_catalog.cc:564] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:20:03.552989 15622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:20:03.553188 15622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:20:03.556911 15290 master_runner.cc:263] No leader master found from master f80e91f3041146ffaf8276642e0fc40b
I20260811 06:20:03.557049 15290 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:20:03.569857 15202 tablet.cc:2369] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:20:03.578787 15575 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:20:03.579051 15575 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.583806 15575 leader_election.cc:288] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fee4cfcaa145918e23119294b6083d (127.22.121.62:40379), f80e91f3041146ffaf8276642e0fc40b (127.22.121.61:42435)
I20260811 06:20:03.584579 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4029c6166f34b68a7072595056a6dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fee4cfcaa145918e23119294b6083d" is_pre_election: true
I20260811 06:20:03.584591 15359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4029c6166f34b68a7072595056a6dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e91f3041146ffaf8276642e0fc40b" is_pre_election: true
I20260811 06:20:03.585131 15236 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4029c6166f34b68a7072595056a6dc6 in term 0.
I20260811 06:20:03.585136 15359 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4029c6166f34b68a7072595056a6dc6 in term 0.
I20260811 06:20:03.585731 15441 leader_election.cc:302] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4029c6166f34b68a7072595056a6dc6, f80e91f3041146ffaf8276642e0fc40b; no voters:
I20260811 06:20:03.586127 15575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:20:03.586297 15575 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:20:03.586405 15575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:03.589749 15575 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.590404 15575 leader_election.cc:288] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [CANDIDATE]: Term 1 election: Requested vote from peers a2fee4cfcaa145918e23119294b6083d (127.22.121.62:40379), f80e91f3041146ffaf8276642e0fc40b (127.22.121.61:42435)
I20260811 06:20:03.590857 15359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4029c6166f34b68a7072595056a6dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e91f3041146ffaf8276642e0fc40b"
I20260811 06:20:03.590875 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e4029c6166f34b68a7072595056a6dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fee4cfcaa145918e23119294b6083d"
I20260811 06:20:03.591049 15359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:03.591075 15236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:03.594165 15236 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4029c6166f34b68a7072595056a6dc6 in term 1.
I20260811 06:20:03.594559 15441 leader_election.cc:302] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fee4cfcaa145918e23119294b6083d, e4029c6166f34b68a7072595056a6dc6; no voters:
I20260811 06:20:03.594393 15359 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4029c6166f34b68a7072595056a6dc6 in term 1.
I20260811 06:20:03.594843 15575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:03.595548 15575 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [term 1 LEADER]: Becoming Leader. State: Replica: e4029c6166f34b68a7072595056a6dc6, State: Running, Role: LEADER
I20260811 06:20:03.596154 15575 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } }
I20260811 06:20:03.599584 15624 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4029c6166f34b68a7072595056a6dc6. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.600013 15624 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:03.601115 15630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:03.603375 15630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:03.615731 15359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [term 1 FOLLOWER]: Refusing update from remote peer e4029c6166f34b68a7072595056a6dc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:03.615877 15236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [term 1 FOLLOWER]: Refusing update from remote peer e4029c6166f34b68a7072595056a6dc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:03.616711 15624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:03.617033 15575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:03.645645 15607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4029c6166f34b68a7072595056a6dc6. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.645977 15607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.648979 15632 mvcc.cc:205] Tried to move back new op lower bound from 7317214018000883712 to 7317214017934540800. Current Snapshot: MvccSnapshot[applied={T|T < 7317214018000883712}]
I20260811 06:20:03.651700 15624 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.651945 15624 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:03.653792 15630 catalog_manager.cc:1347] Generated new cluster ID: de4e0a3d6056493c8dce8e502a77d0c4
I20260811 06:20:03.653889 15630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:03.655566 15635 mvcc.cc:205] Tried to move back new op lower bound from 7317214018000883712 to 7317214017934540800. Current Snapshot: MvccSnapshot[applied={T|T < 7317214018000883712}]
I20260811 06:20:03.658012 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4029c6166f34b68a7072595056a6dc6. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.658300 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.659701 15575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.659976 15575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:03.661468 15633 mvcc.cc:205] Tried to move back new op lower bound from 7317214018000883712 to 7317214017934540800. Current Snapshot: MvccSnapshot[applied={T|T < 7317214018000883712}]
I20260811 06:20:03.667824 15607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.668066 15607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.673400 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4029c6166f34b68a7072595056a6dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fee4cfcaa145918e23119294b6083d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 40379 } } peers { permanent_uuid: "f80e91f3041146ffaf8276642e0fc40b" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 42435 } } peers { permanent_uuid: "e4029c6166f34b68a7072595056a6dc6" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 35897 } } }
I20260811 06:20:03.673691 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:20:03.689018 15630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:20:03.690809 15630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:03.703763 15630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4029c6166f34b68a7072595056a6dc6: Generated new TSK 0
I20260811 06:20:03.704727 15630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:20:04.389758 15582 catalog_manager.cc:1259] Loaded cluster ID: de4e0a3d6056493c8dce8e502a77d0c4
I20260811 06:20:04.389881 15582 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: loading cluster ID for follower catalog manager: success
I20260811 06:20:04.392536 15582 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: acquiring CA information for follower catalog manager: success
I20260811 06:20:04.394364 15582 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2fee4cfcaa145918e23119294b6083d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:20:04.555064 15622 catalog_manager.cc:1259] Loaded cluster ID: de4e0a3d6056493c8dce8e502a77d0c4
I20260811 06:20:04.555181 15622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: loading cluster ID for follower catalog manager: success
I20260811 06:20:04.557586 15622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: acquiring CA information for follower catalog manager: success
I20260811 06:20:04.559453 15622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f80e91f3041146ffaf8276642e0fc40b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:20:04.784384 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 15163
I20260811 06:20:04.785790 15269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:04.891577 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 15163
I20260811 06:20:04.917486 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 15290
I20260811 06:20:04.918582 15395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:05.022621 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 15290
I20260811 06:20:05.049492 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 15410
I20260811 06:20:05.050485 15523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:05.154753 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 15410
2026-08-11T06:20:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4672 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:20:05 dist-test-slave-gjrz krb5kdc[15652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:20:05 dist-test-slave-gjrz krb5kdc[15652](info): set up 2 sockets
Aug 11 06:20:05 dist-test-slave-gjrz krb5kdc[15652](info): commencing operation
krb5kdc: starting...
W20260811 06:20:06.386248 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:20:06 dist-test-slave-gjrz krb5kdc[15652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:20:06.600503 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43929
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:43929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:06.914466 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:06.914721 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:06.914847 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:06.921651 15669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:06.921741 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:06.921816 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:06.921907 15669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:06.921972 15669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:06.931043 15669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:43929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:06.932129 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:06.933288 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:06.944154 15683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:06.944129 15685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:06.944178 15689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:06.945000 15669 server_base.cc:1034] running on GCE node
I20260811 06:20:06.945650 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:06.946841 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:06.948035 15669 hybrid_clock.cc:648] HybridClock initialized: now 1786429206947982 us; error 33 us; skew 500 ppm
I20260811 06:20:06.948325 15669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:06 dist-test-slave-gjrz krb5kdc[15652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429206, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:06.951068 15669 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:06.952555 15669 webserver.cc:460] Webserver started at http://127.22.121.62:36313/ using document root <none> and password file <none>
I20260811 06:20:06.953195 15669 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:06.953315 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:06.953588 15669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:06.956322 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "5b50ecebaa5144d99d8ebc1f96862333"
format_stamp: "Formatted at 2026-08-11 06:20:06 on dist-test-slave-gjrz"
server_key: "d20ff2f768364483b5fb80cc004fde11"
server_key_iv: "52fc10d30e93a36df8cbf0133652a2d0"
server_key_version: "encryptionkey@0"
I20260811 06:20:06.957010 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "5b50ecebaa5144d99d8ebc1f96862333"
format_stamp: "Formatted at 2026-08-11 06:20:06 on dist-test-slave-gjrz"
server_key: "d20ff2f768364483b5fb80cc004fde11"
server_key_iv: "52fc10d30e93a36df8cbf0133652a2d0"
server_key_version: "encryptionkey@0"
I20260811 06:20:06.961678 15669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 06:20:06.964830 15699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:06.965971 15669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:20:06.966136 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "5b50ecebaa5144d99d8ebc1f96862333"
format_stamp: "Formatted at 2026-08-11 06:20:06 on dist-test-slave-gjrz"
server_key: "d20ff2f768364483b5fb80cc004fde11"
server_key_iv: "52fc10d30e93a36df8cbf0133652a2d0"
server_key_version: "encryptionkey@0"
I20260811 06:20:06.966296 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:06.984766 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:06.988253 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:06.988492 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:07.004949 15669 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:43929
I20260811 06:20:07.004980 15785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:43929 every 8 connection(s)
I20260811 06:20:07.006160 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:07.009250 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 15669
I20260811 06:20:07.009482 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:20:07.010002 15787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:07.010363 23012 external_mini_cluster.cc:1428] Setting key f825d8dd421c6ea99fd1aae62a65f43b
I20260811 06:20:07.024137 15787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333: Bootstrap starting.
Aug 11 06:20:07 dist-test-slave-gjrz krb5kdc[15652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:07.027366 15787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:07.028394 15787 log.cc:824] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:07.030932 15787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333: No bootstrap required, opened a new log
I20260811 06:20:07.032555 15794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:07.012152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.22.121.62:43929)
0811 06:20:07.012894 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:07.012905 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:07.012950 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:20:07.014627 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:07.014648 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:07.015009 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:07.015359 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:07.017337 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:07.018549 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:07.020324 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:07.021005 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:07.021142 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:07.025770 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:07.025777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:07.025790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:07.025823 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:07.028840 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:07.029480 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:07.029486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:07.029492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:07.029661 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:07.030030 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:07.030033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:07.030036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:07.030517 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:07.030787 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:07.031227 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 06:20:07.031656 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":331,"threads_started":1}
I20260811 06:20:07.036008 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:20:07.038540 15787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } }
I20260811 06:20:07.038928 15787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:07.039098 15787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b50ecebaa5144d99d8ebc1f96862333, State: Initialized, Role: FOLLOWER
I20260811 06:20:07.039667 15787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } }
I20260811 06:20:07.039878 15787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:07.039985 15787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:07.040112 15787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:07.042114 15787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } }
I20260811 06:20:07.042533 15787 leader_election.cc:302] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b50ecebaa5144d99d8ebc1f96862333; no voters:
I20260811 06:20:07.043004 15787 leader_election.cc:288] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:20:07.043110 15796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:07.043758 15796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [term 1 LEADER]: Becoming Leader. State: Replica: 5b50ecebaa5144d99d8ebc1f96862333, State: Running, Role: LEADER
I20260811 06:20:07.044287 15796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } }
I20260811 06:20:07.044826 15787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:07.046816 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b50ecebaa5144d99d8ebc1f96862333. Latest consensus state: current_term: 1 leader_uuid: "5b50ecebaa5144d99d8ebc1f96862333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } } }
I20260811 06:20:07.047181 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:07.047384 15797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b50ecebaa5144d99d8ebc1f96862333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b50ecebaa5144d99d8ebc1f96862333" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43929 } } }
I20260811 06:20:07.047761 15797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:07.049108 15805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:07.053242 15805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:07.060458 15805 catalog_manager.cc:1347] Generated new cluster ID: 6d6ce9b591f0483999ee0a5e46f66f43
I20260811 06:20:07.060554 15805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:07.090502 15805 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:20:07.092309 15805 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:07.100031 15805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b50ecebaa5144d99d8ebc1f96862333: Generated new TSK 0
I20260811 06:20:07.100939 15805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:20:07.147295 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 15669
I20260811 06:20:07.148263 15778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:07.241951 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 15669
2026-08-11T06:20:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2087 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:20:07 dist-test-slave-gjrz krb5kdc[15826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:20:07 dist-test-slave-gjrz krb5kdc[15826](info): set up 2 sockets
Aug 11 06:20:07 dist-test-slave-gjrz krb5kdc[15826](info): commencing operation
krb5kdc: starting...
W20260811 06:20:08.473340 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:20:08 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:20:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:20:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:20:08.688946 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:41869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:09.000113 15845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:09.000371 15845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:09.000492 15845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:09.007442 15845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:09.007536 15845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:09.007609 15845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:09.007678 15845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:09.007727 15845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:09.017024 15845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:41869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:09.018338 15845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:09.019609 15845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:09.030364 15852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:09.030421 15857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:09.030391 15853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:09.031502 15845 server_base.cc:1034] running on GCE node
I20260811 06:20:09.032193 15845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:09.033468 15845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:09.034662 15845 hybrid_clock.cc:648] HybridClock initialized: now 1786429209034622 us; error 29 us; skew 500 ppm
I20260811 06:20:09.034976 15845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:09 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429209, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:09.041429 15845 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:09.042831 15845 webserver.cc:460] Webserver started at http://127.22.121.62:32825/ using document root <none> and password file <none>
I20260811 06:20:09.043497 15845 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:09.043602 15845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:09.043840 15845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:09.046599 15845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "8964a08bcacf45ff8b39419b05a4e226"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "6cdae3e57aba349c1b4ecbeaf4721300"
server_key_iv: "b8e7546bc60748eda3880700a290aada"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.047331 15845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "8964a08bcacf45ff8b39419b05a4e226"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "6cdae3e57aba349c1b4ecbeaf4721300"
server_key_iv: "b8e7546bc60748eda3880700a290aada"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.052084 15845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:20:09.055235 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:09.056528 15845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:09.056691 15845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "8964a08bcacf45ff8b39419b05a4e226"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "6cdae3e57aba349c1b4ecbeaf4721300"
server_key_iv: "b8e7546bc60748eda3880700a290aada"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.056896 15845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:09.087169 15845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:09.091218 15845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:09.091508 15845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:09.108929 15845 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:41869
I20260811 06:20:09.108964 15955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:41869 every 8 connection(s)
I20260811 06:20:09.110190 15845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:09.113742 15956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:09.116211 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 15845
I20260811 06:20:09.116405 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:20:09.117026 23012 external_mini_cluster.cc:1428] Setting key 46f0c9cf50901eb63164e1c0de58392a
I20260811 06:20:09.125133 15956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap starting.
I20260811 06:20:09.128726 15956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:09.129954 15956 log.cc:824] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:20:09 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:09.132897 15956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: No bootstrap required, opened a new log
I20260811 06:20:09.140334 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:09.118914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.22.121.62:41869)
0811 06:20:09.119692 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:09.119704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:09.119742 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:20:09.121340 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:09.121363 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:09.121720 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:09.122038 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:09.123745 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.124987 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.126794 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.127489 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.127613 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:09.132293 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:09.132305 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:09.132324 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:09.132366 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:09.136155 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:09.136760 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:09.136772 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:09.136780 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:09.137013 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:09.137342 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:09.137347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:09.137350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:09.137994 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:09.138307 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:09.138579 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 06:20:09.138989 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":347,"threads_started":1}
I20260811 06:20:09.142959 15956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:09.143318 15956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:09.143412 15956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Initialized, Role: FOLLOWER
I20260811 06:20:09.143908 15956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:09.144127 15956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:09.144250 15956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:09.144388 15956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:09.146436 15956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:09.146875 15956 leader_election.cc:302] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8964a08bcacf45ff8b39419b05a4e226; no voters:
I20260811 06:20:09.147434 15956 leader_election.cc:288] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:20:09.147565 15964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:09.148185 15964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 LEADER]: Becoming Leader. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Running, Role: LEADER
I20260811 06:20:09.148705 15964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:09.149345 15956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:09.151473 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8964a08bcacf45ff8b39419b05a4e226. Latest consensus state: current_term: 1 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:09.151916 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:09.151667 15966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:09.152110 15966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:09.154101 15977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:09.159314 15977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:09.167155 15977 catalog_manager.cc:1347] Generated new cluster ID: 5b259b0c843341c793b10165b9e813e2
I20260811 06:20:09.167253 15977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:09.189340 15977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:20:09.190549 15977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:09.199509 15977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Generated new TSK 0
I20260811 06:20:09.200277 15977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:20:09.277446 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:09.596547 16000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:09.596789 16000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:09.596887 16000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:09.604147 16000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:09.604238 16000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:09.604348 16000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:20:09.614396 16000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:09.615515 16000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:09.616707 16000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:09.628917 16008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:09.628876 16009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:09.629020 16012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:09.629907 16000 server_base.cc:1034] running on GCE node
I20260811 06:20:09.630445 16000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:09.631283 16000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:09.632491 16000 hybrid_clock.cc:648] HybridClock initialized: now 1786429209632438 us; error 46 us; skew 500 ppm
I20260811 06:20:09.632776 16000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:09 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429209, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:09.635661 16000 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:20:09.636904 16000 webserver.cc:460] Webserver started at http://127.22.121.1:39725/ using document root <none> and password file <none>
I20260811 06:20:09.637528 16000 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:09.637643 16000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:09.637951 16000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:09.640661 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "09d61a09efd545f3af663e067ba0b945"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "69b6738a6bf365bb3523299b5a494477"
server_key_iv: "c6cdd2d7eb8362b65c1bd4de86e173ed"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.641417 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "09d61a09efd545f3af663e067ba0b945"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "69b6738a6bf365bb3523299b5a494477"
server_key_iv: "c6cdd2d7eb8362b65c1bd4de86e173ed"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.646376 16000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:20:09.649636 16022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:09.650928 16000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:20:09.651077 16000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "09d61a09efd545f3af663e067ba0b945"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "69b6738a6bf365bb3523299b5a494477"
server_key_iv: "c6cdd2d7eb8362b65c1bd4de86e173ed"
server_key_version: "encryptionkey@0"
I20260811 06:20:09.651243 16000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:09.663826 16000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:09.667380 16000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:09.667622 16000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:09.668437 16000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:09.669617 16000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:09.669693 16000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:09.669804 16000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:09.669857 16000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:09.699101 16000 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45441
I20260811 06:20:09.699122 16209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45441 every 8 connection(s)
I20260811 06:20:09.700284 16000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:20:09.709080 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 16000
I20260811 06:20:09.709262 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:20:09.709887 23012 external_mini_cluster.cc:1428] Setting key 439c59a041d94f911f0903b170636e5d
Aug 11 06:20:09 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429209, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:09.718256 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:09.702566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:57491 (local address 127.22.121.62:41869)
0811 06:20:09.703024 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:09.703031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:09.703912 (+ 881us) server_negotiation.cc:407] Connection header received
0811 06:20:09.705559 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:09.705563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:09.705629 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:09.705722 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:09.706995 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.707809 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.709261 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.709701 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.709824 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:09.713148 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:09.713153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:09.713156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:09.713186 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:09.715484 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:09.716127 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:09.716131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:09.716133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:09.716249 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:09.716645 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:09.716650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:09.716653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:09.716912 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:09.717042 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:09.717671 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 06:20:09.717893 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":212,"threads_started":1}
I20260811 06:20:09.718998 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:09.703081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:57491)
0811 06:20:09.703762 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:09.703805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:09.705325 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:09.705891 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:09.705898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:09.706341 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:09.706868 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:09.706884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.707919 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.707923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:09.709102 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:09.709119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:09.709915 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:09.709923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:09.710145 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:09.710777 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:09.710806 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:09.712946 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:09.715618 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:09.715627 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:09.715641 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:09.716013 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:09.716350 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:09.716355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:09.716359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:09.716532 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:09.717031 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:09.717039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:09.717326 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 06:20:09.717545 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 06:20:09.717992 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260811 06:20:09.720402 16212 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:09.720674 16212 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:09.721340 16212 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:09.723299 15897 ts_manager.cc:194] Registered new tserver with Master: 09d61a09efd545f3af663e067ba0b945 (127.22.121.1:45441)
I20260811 06:20:09.724748 15897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:57491
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:20:09.760104 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:10.067898 16221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:10.068132 16221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:10.068261 16221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:10.075095 16221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:10.075186 16221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:10.075337 16221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:20:10.084403 16221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:10.085477 16221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:10.086714 16221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:10.099449 16230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:10.099383 16234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:10.099421 16232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:10.100243 16221 server_base.cc:1034] running on GCE node
I20260811 06:20:10.100778 16221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:10.101616 16221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:10.102793 16221 hybrid_clock.cc:648] HybridClock initialized: now 1786429210102743 us; error 41 us; skew 500 ppm
I20260811 06:20:10.103068 16221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:10 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429210, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:10.106019 16221 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:20:10.107379 16221 webserver.cc:460] Webserver started at http://127.22.121.2:37575/ using document root <none> and password file <none>
I20260811 06:20:10.107998 16221 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:10.108115 16221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:10.108397 16221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:10.111145 16221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "3f6ef89cdbe50747307d82817f79cd20"
server_key_iv: "8a82b911ad10b2d373cf644dcbb69f76"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.111918 16221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "3f6ef89cdbe50747307d82817f79cd20"
server_key_iv: "8a82b911ad10b2d373cf644dcbb69f76"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.116652 16221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:20:10.119971 16246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.121240 16221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:10.121384 16221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "3f6ef89cdbe50747307d82817f79cd20"
server_key_iv: "8a82b911ad10b2d373cf644dcbb69f76"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.121542 16221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:10.139509 16221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:10.142897 16221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:10.143136 16221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:10.143975 16221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:10.145181 16221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:10.145257 16221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.145371 16221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:10.145424 16221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.175374 16221 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:40045
I20260811 06:20:10.175407 16437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:40045 every 8 connection(s)
I20260811 06:20:10.176489 16221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:20:10.177923 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 16221
I20260811 06:20:10.178089 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:20:10.178630 23012 external_mini_cluster.cc:1428] Setting key 1544d2b6f1cf2d6d1a57a8ab5553e70a
Aug 11 06:20:10 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429210, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:10.195329 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.178948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49727 (local address 127.22.121.62:41869)
0811 06:20:10.179116 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.179120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.180520 (+ 1400us) server_negotiation.cc:407] Connection header received
0811 06:20:10.182332 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.182338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.182414 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.182500 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:10.183971 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.184763 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.186263 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.186679 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.186790 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.190029 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:10.190034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:10.190037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:10.190075 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:10.192472 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.193268 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.193271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.193273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.193398 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.193835 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.193837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.193839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.194073 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:10.194176 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.194889 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.195046 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:20:10.196599 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.179507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:49727)
0811 06:20:10.180363 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.180419 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.182082 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.182738 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.182746 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.183228 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:10.183795 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.183812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.184907 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.184911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.186127 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.186139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.186894 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.186901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.187117 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.187730 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:10.187752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:10.189762 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:10.192617 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:10.192626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:10.192642 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:10.193167 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:10.193494 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:10.193498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:10.193502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:10.193743 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:10.194187 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:10.194195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:10.194547 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.194829 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.195311 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":324,"threads_started":1}
I20260811 06:20:10.197878 16438 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:10.198170 16438 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:10.198966 16438 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:10.200469 15897 ts_manager.cc:194] Registered new tserver with Master: e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045)
I20260811 06:20:10.201298 15897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:49727
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:20:10.232139 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:10.541323 16445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:10.541579 16445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:10.541718 16445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:10.548851 16445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:10.548976 16445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:10.549115 16445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:20:10.558576 16445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:10.559653 16445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:10.560894 16445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:10.572819 16458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:10.572870 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:10.572889 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:10.573658 16445 server_base.cc:1034] running on GCE node
I20260811 06:20:10.574179 16445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:10.574982 16445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:10.576176 16445 hybrid_clock.cc:648] HybridClock initialized: now 1786429210576132 us; error 28 us; skew 500 ppm
I20260811 06:20:10.576467 16445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:10 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429210, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:10.579385 16445 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:20:10.580618 16445 webserver.cc:460] Webserver started at http://127.22.121.3:34359/ using document root <none> and password file <none>
I20260811 06:20:10.581184 16445 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:10.581280 16445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:10.581524 16445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:10.584208 16445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "1b7f5ada8de63b690feb76b39884bdfd"
server_key_iv: "3a4dc29e48ef25089915ea5fb813d230"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.584909 16445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "1b7f5ada8de63b690feb76b39884bdfd"
server_key_iv: "3a4dc29e48ef25089915ea5fb813d230"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.589561 16445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:20:10.592798 16469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.594002 16445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:20:10.594154 16445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "1b7f5ada8de63b690feb76b39884bdfd"
server_key_iv: "3a4dc29e48ef25089915ea5fb813d230"
server_key_version: "encryptionkey@0"
I20260811 06:20:10.594334 16445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:10.610445 16445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:10.614915 16445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:10.615173 16445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:10.616062 16445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:10.617308 16445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:10.617388 16445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.617503 16445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:10.617555 16445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:10.647316 16445 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42009
I20260811 06:20:10.647356 16662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42009 every 8 connection(s)
I20260811 06:20:10.648455 16445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:20:10.650508 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 16445
I20260811 06:20:10.650692 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:20:10.651173 23012 external_mini_cluster.cc:1428] Setting key 315570f0a7cc114325c15c99b2ae97d7
Aug 11 06:20:10 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429210, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:10.666587 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.650822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:60667 (local address 127.22.121.62:41869)
0811 06:20:10.651025 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.651031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.652304 (+ 1273us) server_negotiation.cc:407] Connection header received
0811 06:20:10.654208 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.654212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.654279 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.654381 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:10.655791 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.656538 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.657958 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.658336 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.658428 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.661510 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:10.661514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:10.661517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:10.661548 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:10.663836 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.664483 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.664487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.664489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.664604 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.665004 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.665008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.665010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.665264 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:10.665396 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.666033 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.666246 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:20:10.667431 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.651397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:60667)
0811 06:20:10.652152 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.652203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.653937 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.654544 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.654550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.655047 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:10.655648 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.655666 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.656661 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.656664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.657853 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.657864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.658500 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.658504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.658682 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.659244 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:10.659304 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:10.661336 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:10.663953 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:10.663959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:10.663969 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:10.664342 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:10.664719 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:10.664723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:10.664725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:10.664888 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:10.665387 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:10.665394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:10.665677 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.665898 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.666336 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":294,"threads_started":1}
I20260811 06:20:10.668665 16663 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:10.668915 16663 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:10.669559 16663 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:10.670794 15897 ts_manager.cc:194] Registered new tserver with Master: 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:10.671571 15897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:60667
I20260811 06:20:10.677687 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:20:10.692679 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.681906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.22.121.62:41869)
0811 06:20:10.682049 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.682053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.682497 (+ 444us) server_negotiation.cc:407] Connection header received
0811 06:20:10.682614 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.682617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.682668 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.682734 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:10.684044 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.684716 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.686049 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.686460 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.686547 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.688538 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:10.688543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:10.688546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:10.688573 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:10.690685 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.691229 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.691232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.691234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.691388 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.691675 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.691678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.691680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.691961 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:10.692140 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.692332 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.692472 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:20:10.696017 15897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:20:10.704701 15897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:20:10.725441 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.715135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:40045 (local address 127.0.0.1:41508)
0811 06:20:10.715927 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.715970 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.716147 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.717206 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.717214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.717247 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:10.717643 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.717661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.719705 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.719711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.722189 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.722206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.724453 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.724460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.724688 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.724741 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.724865 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.725046 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":842,"threads_started":1}
I20260811 06:20:10.726188 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.715235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.22.121.2:40045)
0811 06:20:10.715947 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.715955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.716362 (+ 407us) server_negotiation.cc:407] Connection header received
0811 06:20:10.716474 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.716481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.716739 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.716935 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:10.717844 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.719505 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.722400 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.723964 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.724102 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.724289 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.725665 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.725842 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":427,"threads_started":1}
I20260811 06:20:10.727381 16212 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:10.733227 16324 tablet_service.cc:1467] Processing CreateTablet for tablet d9d557bd768146999d907c984fec4497 (DEFAULT_TABLE table=test-table [id=77ef52e71d7744ea92ea5109b20c1d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:10.734563 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.715776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45441 (local address 127.0.0.1:53890)
0811 06:20:10.726490 (+ 10714us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.726509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.726618 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.727940 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.727945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.727973 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:10.728149 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.728159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.730217 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.730223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.732071 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.732086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.733875 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.733881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.734069 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.734102 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.734167 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.734291 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10616,"thread_start_us":17832,"threads_started":1}
I20260811 06:20:10.734694 16324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d557bd768146999d907c984fec4497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:10.735638 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.716310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.22.121.1:45441)
0811 06:20:10.727259 (+ 10949us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.727287 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.727316 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:20:10.727415 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.727423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.727729 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.728301 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:10.728546 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.730045 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.732282 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.733695 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.734873 (+ 1178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.735098 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.735165 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.735334 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10790,"thread_start_us":10661,"threads_started":1}
I20260811 06:20:10.735994 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.716736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42009 (local address 127.0.0.1:45696)
0811 06:20:10.727982 (+ 11246us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.727998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.728119 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.728985 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.728990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.729014 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:10.729412 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.729423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.731326 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.731332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.733346 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.733361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.735251 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.735257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.735455 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.735487 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.735567 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.735689 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11134,"thread_start_us":12441,"threads_started":1}
I20260811 06:20:10.736850 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.717247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.22.121.3:42009)
0811 06:20:10.727605 (+ 10358us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.727612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.728239 (+ 627us) server_negotiation.cc:407] Connection header received
0811 06:20:10.728333 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.728339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.728538 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.728719 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:10.729629 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.731148 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.733827 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.735058 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.735179 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.735423 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.735622 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.735771 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10201,"thread_start_us":11780,"threads_started":1}
I20260811 06:20:10.738775 16098 tablet_service.cc:1467] Processing CreateTablet for tablet d9d557bd768146999d907c984fec4497 (DEFAULT_TABLE table=test-table [id=77ef52e71d7744ea92ea5109b20c1d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:10.739845 16098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d557bd768146999d907c984fec4497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:10.740120 16558 tablet_service.cc:1467] Processing CreateTablet for tablet d9d557bd768146999d907c984fec4497 (DEFAULT_TABLE table=test-table [id=77ef52e71d7744ea92ea5109b20c1d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:10.741434 16558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d557bd768146999d907c984fec4497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:10.749784 16696 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Bootstrap starting.
I20260811 06:20:10.750984 16697 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Bootstrap starting.
I20260811 06:20:10.753404 16696 tablet_bootstrap.cc:654] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:10.754295 16697 tablet_bootstrap.cc:654] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:10.754534 16696 log.cc:824] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:10.755566 16697 log.cc:824] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:10.756888 16698 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Bootstrap starting.
I20260811 06:20:10.758441 16697 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: No bootstrap required, opened a new log
I20260811 06:20:10.758666 16697 ts_tablet_manager.cc:1397] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 06:20:10.759696 16696 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: No bootstrap required, opened a new log
I20260811 06:20:10.759950 16696 ts_tablet_manager.cc:1397] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 06:20:10.760879 16698 tablet_bootstrap.cc:654] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:10.762084 16698 log.cc:824] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:10.765162 16698 tablet_bootstrap.cc:492] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: No bootstrap required, opened a new log
I20260811 06:20:10.765403 16698 ts_tablet_manager.cc:1397] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:20:10.766204 16697 raft_consensus.cc:348] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.766513 16697 raft_consensus.cc:374] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:10.766607 16697 raft_consensus.cc:729] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d61a09efd545f3af663e067ba0b945, State: Initialized, Role: FOLLOWER
I20260811 06:20:10.767166 16697 consensus_queue.cc:260] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.769546 16696 raft_consensus.cc:348] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.769968 16696 raft_consensus.cc:374] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:10.770069 16696 raft_consensus.cc:729] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e46cf47ddffb46d39fda2cc9a36a4bf0, State: Initialized, Role: FOLLOWER
I20260811 06:20:10.770637 16696 consensus_queue.cc:260] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.771765 16438 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:10.772384 16696 ts_tablet_manager.cc:1428] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 06:20:10.773129 16697 ts_tablet_manager.cc:1428] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 06:20:10.775665 16698 raft_consensus.cc:348] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.776041 16698 raft_consensus.cc:374] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:10.776163 16698 raft_consensus.cc:729] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f23e6cc87cc426ca8d3f1d15bf0263e, State: Initialized, Role: FOLLOWER
I20260811 06:20:10.776706 16698 consensus_queue.cc:260] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.777573 16663 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:10.777947 16698 ts_tablet_manager.cc:1428] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 06:20:10.813319 16706 raft_consensus.cc:482] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:20:10.813520 16706 raft_consensus.cc:504] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.814867 16706 leader_election.cc:288] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045), 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:10.821394 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.815174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:60981 (local address 127.22.121.2:40045)
0811 06:20:10.815395 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.815403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.815851 (+ 448us) server_negotiation.cc:407] Connection header received
0811 06:20:10.816018 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.816024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.816098 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.816181 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:10.816847 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.817740 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.819649 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.820504 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.820623 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.820673 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.821034 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.821155 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:20:10.821436 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.815166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:40045 (local address 127.22.121.1:60981)
0811 06:20:10.815745 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.815765 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.815896 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.816327 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.816331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.816352 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:10.816709 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.816723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.817860 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.817866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.819503 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.819518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.820696 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.820702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.820868 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.820901 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.821004 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.821121 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":35,"thread_start_us":335,"threads_started":1}
I20260811 06:20:10.821755 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.816087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43441 (local address 127.22.121.3:42009)
0811 06:20:10.816255 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.816260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.816518 (+ 258us) server_negotiation.cc:407] Connection header received
0811 06:20:10.816648 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.816653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.816719 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.816810 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:10.817360 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.818213 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.820013 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.820837 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.820984 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.821046 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.821414 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.821542 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:20:10.821857 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.815873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42009 (local address 127.22.121.1:43441)
0811 06:20:10.816424 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:10.816440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:10.816542 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:10.816841 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:10.816846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:10.816869 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:10.817216 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.817228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.818351 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.818355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.819860 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:10.819875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.821058 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.821065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:10.821240 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.821275 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:20:10.821370 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:20:10.821493 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":1,"thread_start_us":259,"threads_started":1}
I20260811 06:20:10.822304 16359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d557bd768146999d907c984fec4497" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" is_pre_election: true
I20260811 06:20:10.822710 16359 raft_consensus.cc:2454] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 0.
I20260811 06:20:10.822667 16588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d557bd768146999d907c984fec4497" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" is_pre_election: true
I20260811 06:20:10.823100 16588 raft_consensus.cc:2454] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 0.
I20260811 06:20:10.823297 16029 leader_election.cc:302] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d61a09efd545f3af663e067ba0b945, e46cf47ddffb46d39fda2cc9a36a4bf0; no voters:
I20260811 06:20:10.823626 16706 raft_consensus.cc:2781] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:20:10.823748 16706 raft_consensus.cc:482] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:20:10.823840 16706 raft_consensus.cc:3037] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:10.826004 16706 raft_consensus.cc:504] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.826508 16706 leader_election.cc:288] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 election: Requested vote from peers e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045), 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:10.826918 16359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d557bd768146999d907c984fec4497" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
I20260811 06:20:10.827004 16588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d557bd768146999d907c984fec4497" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
I20260811 06:20:10.827111 16359 raft_consensus.cc:3037] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:10.827169 16588 raft_consensus.cc:3037] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:10.830142 16588 raft_consensus.cc:2454] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 1.
I20260811 06:20:10.830142 16359 raft_consensus.cc:2454] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 1.
I20260811 06:20:10.830487 16029 leader_election.cc:302] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d61a09efd545f3af663e067ba0b945, e46cf47ddffb46d39fda2cc9a36a4bf0; no voters:
I20260811 06:20:10.830745 16706 raft_consensus.cc:2781] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:10.831127 16706 raft_consensus.cc:686] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 1 LEADER]: Becoming Leader. State: Replica: 09d61a09efd545f3af663e067ba0b945, State: Running, Role: LEADER
I20260811 06:20:10.831517 16706 consensus_queue.cc:237] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } }
I20260811 06:20:10.835179 15897 catalog_manager.cc:5495] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d61a09efd545f3af663e067ba0b945 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "09d61a09efd545f3af663e067ba0b945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } health_report { overall_health: HEALTHY } } }
I20260811 06:20:10.862969 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.858081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.22.121.1:45441)
0811 06:20:10.858264 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.858270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.858373 (+ 103us) server_negotiation.cc:407] Connection header received
0811 06:20:10.858552 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.858556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.858624 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.858730 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:10.859038 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.859741 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.861233 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.861612 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.861696 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.862144 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:10.862559 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:20:10.862669 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.862704 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.862760 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:20:10.870420 16588 raft_consensus.cc:1261] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Refusing update from remote peer 09d61a09efd545f3af663e067ba0b945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:10.870431 16359 raft_consensus.cc:1261] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Refusing update from remote peer 09d61a09efd545f3af663e067ba0b945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:10.871345 16719 consensus_queue.cc:1035] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:10.871663 16706 consensus_queue.cc:1035] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:10.894690 15897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:10.894915 15897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:10.898030 15897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Sending DeleteTablet for 3 replicas of tablet d9d557bd768146999d907c984fec4497
I20260811 06:20:10.899145 16558 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d557bd768146999d907c984fec4497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:10 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:45696
I20260811 06:20:10.899127 16324 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d557bd768146999d907c984fec4497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:10 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:41508
I20260811 06:20:10.899302 16098 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d557bd768146999d907c984fec4497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:10 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:53890
I20260811 06:20:10.902932 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 16000
I20260811 06:20:10.910081 16743 tablet_replica.cc:330] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: stopping tablet replica
I20260811 06:20:10.911111 16743 raft_consensus.cc:2229] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:20:10.912390 16743 raft_consensus.cc:2258] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:10.916608 16743 ts_tablet_manager.cc:1905] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:10.918499 16742 tablet_replica.cc:330] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: stopping tablet replica
I20260811 06:20:10.918865 16742 raft_consensus.cc:2229] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:10.922794 16742 raft_consensus.cc:2258] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:10.923557 16740 tablet_replica.cc:330] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: stopping tablet replica
I20260811 06:20:10.923915 16740 raft_consensus.cc:2229] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:10.924640 16740 raft_consensus.cc:2258] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:10.925752 16743 ts_tablet_manager.cc:1918] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:10.925943 16743 log.cc:1196] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/d9d557bd768146999d907c984fec4497
I20260811 06:20:10.926131 16742 ts_tablet_manager.cc:1905] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:10.926740 16743 ts_tablet_manager.cc:1939] T d9d557bd768146999d907c984fec4497 P 09d61a09efd545f3af663e067ba0b945: Deleting consensus metadata
I20260811 06:20:10.928440 15870 catalog_manager.cc:4841] TS 09d61a09efd545f3af663e067ba0b945 (127.22.121.1:45441): tablet d9d557bd768146999d907c984fec4497 (table test-table [id=77ef52e71d7744ea92ea5109b20c1d60]) successfully deleted
Aug 11 06:20:10 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:20:10.929096 16740 ts_tablet_manager.cc:1905] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:10.935081 16742 ts_tablet_manager.cc:1918] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:10.935235 16742 log.cc:1196] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/d9d557bd768146999d907c984fec4497
I20260811 06:20:10.935566 16740 ts_tablet_manager.cc:1918] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:10.935727 16740 log.cc:1196] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/d9d557bd768146999d907c984fec4497
I20260811 06:20:10.935787 16742 ts_tablet_manager.cc:1939] T d9d557bd768146999d907c984fec4497 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting consensus metadata
I20260811 06:20:10.936239 16740 ts_tablet_manager.cc:1939] T d9d557bd768146999d907c984fec4497 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting consensus metadata
I20260811 06:20:10.937386 15869 catalog_manager.cc:4841] TS 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009): tablet d9d557bd768146999d907c984fec4497 (table test-table [id=77ef52e71d7744ea92ea5109b20c1d60]) successfully deleted
I20260811 06:20:10.937472 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:10.904029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53910 (local address 127.22.121.1:45441)
0811 06:20:10.904187 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:10.904193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:10.915599 (+ 11406us) server_negotiation.cc:407] Connection header received
0811 06:20:10.916008 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:10.916014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:10.916090 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:10.916168 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:10.919386 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.920699 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.923445 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:10.924145 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:10.924239 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:10.929722 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:10.929737 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:10.929752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:10.929793 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:10.933626 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.934519 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.934527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.934531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.934697 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:10.935242 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:10.935247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:10.935250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:10.935929 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:10.936211 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:10.936897 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 06:20:10.937131 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:20:10.937656 15869 catalog_manager.cc:4841] TS e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045): tablet d9d557bd768146999d907c984fec4497 (table test-table [id=77ef52e71d7744ea92ea5109b20c1d60]) successfully deleted
I20260811 06:20:10.938946 16203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:11.033115 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 16000
I20260811 06:20:11.058552 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 16221
Aug 11 06:20:11 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:20:11.073033 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:11.059310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.22.121.2:40045)
0811 06:20:11.059481 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:11.059486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:11.059505 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:20:11.059629 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:11.059632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:11.059697 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:11.059780 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:11.061060 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:11.061848 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:11.063589 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:11.063978 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:11.064073 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:11.068024 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:11.068037 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:11.068062 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:11.068099 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:11.070653 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:11.071295 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:11.071305 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:11.071308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:11.071436 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:11.071802 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:11.071805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:11.071807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:11.072188 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:11.072397 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:11.072625 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:20:11.072779 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:20:11.073694 16429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:11.159099 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 16221
I20260811 06:20:11.179385 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 16445
Aug 11 06:20:11 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:20:11.193625 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:11.180141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.22.121.3:42009)
0811 06:20:11.180347 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:11.180353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:11.180376 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:20:11.180463 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:11.180467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:11.180540 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:11.180640 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:11.181925 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:11.182641 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:11.183978 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:11.184398 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:11.184498 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:11.188700 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:11.188715 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:11.188734 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:11.188764 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:11.191181 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:11.191821 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:11.191828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:11.191831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:11.191966 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:11.192254 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:11.192256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:11.192258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:11.192577 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:11.192790 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:11.193125 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 06:20:11.193339 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:20:11.194224 16655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:11.279069 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 16445
I20260811 06:20:11.297240 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 15845
I20260811 06:20:11.298348 15948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:11.404529 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 15845
I20260811 06:20:11.432865 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--webserver_interface=127.22.121.62
--webserver_port=32825
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:41869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:11.754798 16752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:11.755082 16752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:11.755220 16752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:11.762075 16752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:11.762177 16752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:11.762255 16752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:11.762334 16752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:11.762395 16752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:11.771536 16752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:41869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:11.772673 16752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:11.773881 16752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:11.797484 16770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:11.797523 16765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:11.797492 16766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:11.798385 16752 server_base.cc:1034] running on GCE node
I20260811 06:20:11.799152 16752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:11.800631 16752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:11.801826 16752 hybrid_clock.cc:648] HybridClock initialized: now 1786429211801783 us; error 33 us; skew 500 ppm
I20260811 06:20:11.802241 16752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:11 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429211, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:11.805567 16752 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:11.807112 16752 webserver.cc:460] Webserver started at http://127.22.121.62:32825/ using document root <none> and password file <none>
I20260811 06:20:11.807812 16752 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:11.807940 16752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:11.813514 16752 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:20:11.816253 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:11.817489 16752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:11.817643 16752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "8964a08bcacf45ff8b39419b05a4e226"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "6cdae3e57aba349c1b4ecbeaf4721300"
server_key_iv: "b8e7546bc60748eda3880700a290aada"
server_key_version: "encryptionkey@0"
I20260811 06:20:11.818635 16752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:11.837414 16752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:11.841018 16752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:11.841257 16752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:11.858316 16752 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:41869
I20260811 06:20:11.858335 16874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:41869 every 8 connection(s)
I20260811 06:20:11.859692 16752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:11.863776 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 16752
I20260811 06:20:11.864547 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:45441
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=39725
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:11.866282 16876 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:20:11.869428 16876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap starting.
I20260811 06:20:11.888793 16876 log.cc:824] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:11.900338 16876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:20:11.900800 16876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap complete.
I20260811 06:20:11.909469 16876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:11.910243 16876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Initialized, Role: FOLLOWER
I20260811 06:20:11.910775 16876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:11.910997 16876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:11.911111 16876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:11.911290 16876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:20:11.914850 16876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:11.915387 16876 leader_election.cc:302] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8964a08bcacf45ff8b39419b05a4e226; no voters:
I20260811 06:20:11.915951 16876 leader_election.cc:288] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:20:11.916081 16887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:20:11.917212 16887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 LEADER]: Becoming Leader. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Running, Role: LEADER
I20260811 06:20:11.917804 16887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:11.918130 16876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:11.920257 16894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8964a08bcacf45ff8b39419b05a4e226. Latest consensus state: current_term: 2 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:11.920646 16894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:11.925025 16889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:11.925356 16889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:11.925866 16902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:11.934366 16902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:11.935761 16902 catalog_manager.cc:1259] Loaded cluster ID: 5b259b0c843341c793b10165b9e813e2
I20260811 06:20:11.935896 16902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:11.938974 16902 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 06:20:11.956156 16902 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 06:20:11.964522 16902 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:11.966151 16902 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Loaded TSK: 0
I20260811 06:20:11.967645 16902 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:20:12.193763 16882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:12.194005 16882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:12.194128 16882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:12.201041 16882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:12.201133 16882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:12.201264 16882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:20:12.210391 16882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:45441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=39725
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:12.211562 16882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:12.212782 16882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:12.226135 16929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:12.226168 16930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:12.227218 16933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:12.227813 16882 server_base.cc:1034] running on GCE node
I20260811 06:20:12.228370 16882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:12.229260 16882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:12.230473 16882 hybrid_clock.cc:648] HybridClock initialized: now 1786429212230426 us; error 40 us; skew 500 ppm
I20260811 06:20:12.230753 16882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:12 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429212, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:12.233708 16882 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:20:12.235132 16882 webserver.cc:460] Webserver started at http://127.22.121.1:39725/ using document root <none> and password file <none>
I20260811 06:20:12.235769 16882 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:12.235889 16882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:12.240850 16882 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:20:12.243474 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.244786 16882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:12.244937 16882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "09d61a09efd545f3af663e067ba0b945"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "69b6738a6bf365bb3523299b5a494477"
server_key_iv: "c6cdd2d7eb8362b65c1bd4de86e173ed"
server_key_version: "encryptionkey@0"
I20260811 06:20:12.245843 16882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:12.262128 16882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:12.265818 16882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:12.266057 16882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:12.266995 16882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:12.268263 16882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:12.268368 16882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.268473 16882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:12.268544 16882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.298276 16882 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45441
I20260811 06:20:12.298352 17130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45441 every 8 connection(s)
I20260811 06:20:12.299691 16882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:20:12.308005 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 16882
I20260811 06:20:12.308598 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:40045
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=37575
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:20:12 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429212, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:12.322397 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:12.302439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:58409)
0811 06:20:12.303216 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:12.303277 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:12.305164 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:12.306385 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:12.306392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:12.306856 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:12.307412 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:12.307431 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.309124 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.309130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:12.310865 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:12.310879 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.311866 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.311872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:12.312128 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:12.312785 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:12.312818 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:12.315119 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:12.318357 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:12.318365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:12.318378 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:12.318770 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:12.319191 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:12.319196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:12.319200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:12.319429 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:12.320115 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:12.320122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:12.320419 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 06:20:12.320638 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 06:20:12.321141 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":304,"threads_started":1}
I20260811 06:20:12.322396 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:12.302372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:58409 (local address 127.22.121.62:41869)
0811 06:20:12.303071 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:12.303084 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:12.303384 (+ 300us) server_negotiation.cc:407] Connection header received
0811 06:20:12.305643 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:12.305655 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:12.305968 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:12.306246 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:12.307614 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.308960 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.310981 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.311647 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.311773 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:12.315395 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:12.315403 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:12.315417 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:12.315451 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:12.318231 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:12.318901 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:12.318907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:12.318913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:12.319086 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:12.319536 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:12.319540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:12.319543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:12.319973 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:12.320147 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:12.320879 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 06:20:12.321245 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":354,"threads_started":1}
I20260811 06:20:12.325345 17135 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:12.325608 17135 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:12.326329 17135 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:12.328420 16804 ts_manager.cc:194] Registered new tserver with Master: 09d61a09efd545f3af663e067ba0b945 (127.22.121.1:45441)
I20260811 06:20:12.329824 16804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:58409
W20260811 06:20:12.624503 17142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:12.624748 17142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:12.624872 17142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:12.631733 17142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:12.631834 17142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:12.631969 17142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:20:12.640883 17142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:40045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=37575
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:12.641969 17142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:12.643189 17142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:12.655177 17149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:12.655225 17150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:12.655225 17153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:12.656121 17142 server_base.cc:1034] running on GCE node
I20260811 06:20:12.656633 17142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:12.657433 17142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:12.658632 17142 hybrid_clock.cc:648] HybridClock initialized: now 1786429212658578 us; error 42 us; skew 500 ppm
I20260811 06:20:12.658905 17142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:12 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429212, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:12.661713 17142 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:20:12.663045 17142 webserver.cc:460] Webserver started at http://127.22.121.2:37575/ using document root <none> and password file <none>
I20260811 06:20:12.663681 17142 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:12.663805 17142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:12.668567 17142 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:20:12.670958 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.672147 17142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:20:12.672286 17142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "3f6ef89cdbe50747307d82817f79cd20"
server_key_iv: "8a82b911ad10b2d373cf644dcbb69f76"
server_key_version: "encryptionkey@0"
I20260811 06:20:12.673167 17142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:12.696739 17142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:12.700088 17142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:12.700312 17142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:12.701154 17142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:12.702348 17142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:12.702448 17142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.702551 17142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:12.702620 17142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:12.732183 17142 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:40045
I20260811 06:20:12.732254 17338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:40045 every 8 connection(s)
I20260811 06:20:12.733445 17142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:20:12.743438 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 17142
I20260811 06:20:12.744094 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:42009
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=34359
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:20:12 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429212, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:12.751621 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:12.735681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:57235 (local address 127.22.121.62:41869)
0811 06:20:12.735832 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:12.735836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:12.736894 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 06:20:12.738647 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:12.738654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:12.738731 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:12.738820 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:12.740259 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.740946 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.742464 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.742902 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.743011 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:12.746241 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:12.746246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:12.746250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:12.746281 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:12.748599 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:12.749351 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:12.749356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:12.749360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:12.749483 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:12.749917 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:12.749922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:12.749925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:12.750224 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:12.750373 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:12.751014 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 06:20:12.751252 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:20:12.752468 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:12.736083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:57235)
0811 06:20:12.736740 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:12.736783 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:12.738361 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:12.739053 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:12.739060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:12.739583 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:12.740095 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:12.740111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.741102 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.741106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:12.742313 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:12.742326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:12.743113 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:12.743121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:12.743384 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:12.743996 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:12.744017 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:12.746046 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:12.748726 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:12.748735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:12.748750 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:12.749242 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:12.749609 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:12.749614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:12.749618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:12.749788 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:12.750365 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:12.750372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:12.750668 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 06:20:12.750890 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 06:20:12.751365 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260811 06:20:12.753801 17339 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:12.754065 17339 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:12.754714 17339 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:12.756067 16804 ts_manager.cc:194] Registered new tserver with Master: e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045)
I20260811 06:20:12.756829 16804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:57235
W20260811 06:20:13.056171 17344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:13.056421 17344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:13.056565 17344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:13.063418 17344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:13.063524 17344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:13.063673 17344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:20:13.072646 17344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:42009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=34359
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:13.073745 17344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:13.074978 17344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:13.087438 17355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:13.087442 17353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:13.087539 17361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:13.088166 17344 server_base.cc:1034] running on GCE node
I20260811 06:20:13.088729 17344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:13.089634 17344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:13.090832 17344 hybrid_clock.cc:648] HybridClock initialized: now 1786429213090784 us; error 45 us; skew 500 ppm
I20260811 06:20:13.091122 17344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:13 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429213, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:13.093987 17344 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:20:13.095561 17344 webserver.cc:460] Webserver started at http://127.22.121.3:34359/ using document root <none> and password file <none>
I20260811 06:20:13.096186 17344 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:13.096318 17344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:13.101212 17344 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:20:13.103780 17372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:13.104971 17344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:13.105110 17344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "1b7f5ada8de63b690feb76b39884bdfd"
server_key_iv: "3a4dc29e48ef25089915ea5fb813d230"
server_key_version: "encryptionkey@0"
I20260811 06:20:13.105986 17344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:13.117835 17344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:13.121457 17344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:13.121699 17344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:13.122573 17344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:13.123795 17344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:13.123898 17344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:13.123999 17344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:13.124071 17344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:13.153502 17344 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42009
I20260811 06:20:13.153590 17557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42009 every 8 connection(s)
I20260811 06:20:13.154814 17344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:20:13.158279 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 17344
Aug 11 06:20:13 dist-test-slave-gjrz krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429213, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:13.175706 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.159245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.22.121.62:41869)
0811 06:20:13.161582 (+ 2337us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.161586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.161599 (+ 13us) server_negotiation.cc:407] Connection header received
0811 06:20:13.161636 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.161638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.161682 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.161770 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:13.163506 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.164558 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.166025 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.166595 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.166719 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.169682 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:13.169689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:13.169694 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:13.169730 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:13.172908 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.173614 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.173620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.173623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.173796 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.174184 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.174189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.174191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.174486 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:13.174663 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.175060 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.175307 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2255,"thread_start_us":2130,"threads_started":1}
I20260811 06:20:13.175931 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.157110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:48003 (local address 127.22.121.62:41869)
0811 06:20:13.157269 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.157273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.158430 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 06:20:13.161071 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.161075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.161143 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.161230 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:13.162697 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.163822 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.165587 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.166166 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.166252 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.170229 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:13.170233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:13.170236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:13.170275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:13.173076 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.173764 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.173768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.173770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.173902 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.174325 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.174329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.174331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.174593 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:13.174728 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.175494 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.175679 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:20:13.177052 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.157499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:48003)
0811 06:20:13.158248 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.158312 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.160740 (+ 2428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.161496 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.161507 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.161990 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:13.162530 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.162547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.163980 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.163986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.165433 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.165454 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.166366 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.166374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.166655 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.167337 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:13.167369 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:13.170017 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:13.173204 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:13.173210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:13.173220 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:13.173642 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:13.174008 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:13.174012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:13.174014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:13.174213 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:13.174724 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:13.174729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:13.175029 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.175397 (+ 368us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.175902 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":268,"threads_started":1}
I20260811 06:20:13.178319 17560 heartbeater.cc:347] Connected to a master server at 127.22.121.62:41869
I20260811 06:20:13.178637 17560 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:13.179394 17560 heartbeater.cc:511] Master 127.22.121.62:41869 requested a full tablet report, sending...
I20260811 06:20:13.180677 16803 ts_manager.cc:194] Registered new tserver with Master: 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:13.181471 16803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:48003
I20260811 06:20:13.190737 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:20:13.196676 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.191718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.22.121.62:41869)
0811 06:20:13.191859 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.191863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.192230 (+ 367us) server_negotiation.cc:407] Connection header received
0811 06:20:13.192391 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.192395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.192514 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.192623 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:13.192935 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.193592 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.194980 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.195411 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.195502 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.195885 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:13.196290 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:20:13.196386 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.196454 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.196509 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:20:13.197479 16803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:20:13.204392 16803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:20:13.239096 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.215038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.22.121.1:45441)
0811 06:20:13.230411 (+ 15373us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.230419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.230451 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:20:13.230545 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.230553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.230785 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.231935 (+ 1150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:13.232735 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.234177 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.236584 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.237980 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.238135 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.238324 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.238596 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.238755 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15202,"thread_start_us":15093,"threads_started":1}
I20260811 06:20:13.239125 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.214759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45441 (local address 127.0.0.1:53920)
0811 06:20:13.215758 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.215802 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.215949 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.231963 (+ 16014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.231974 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.232005 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:13.232491 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.232505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.234326 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.234332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.236441 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.236454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.238190 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.238196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.238368 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.238422 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.238543 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.238717 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":134,"thread_start_us":17392,"threads_started":1}
I20260811 06:20:13.239710 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.215028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:40045 (local address 127.0.0.1:41520)
0811 06:20:13.231445 (+ 16417us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.231462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.231564 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.232543 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.232547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.232571 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:13.232912 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.232923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.234589 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.234593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.237024 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.237039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.238859 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.238866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.239050 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.239087 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.239178 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.239321 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16303,"mutex_wait_us":60,"thread_start_us":925,"threads_started":1}
I20260811 06:20:13.239857 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.215774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.22.121.2:40045)
0811 06:20:13.231993 (+ 16219us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.232001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.232044 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:20:13.232134 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.232140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.232354 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.232532 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:13.233095 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.234462 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.237213 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.238336 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.238449 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.238695 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.239365 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.239528 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16059,"thread_start_us":15959,"threads_started":1}
I20260811 06:20:13.240152 17581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.216702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42009 (local address 127.0.0.1:45716)
0811 06:20:13.232350 (+ 15648us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.232364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.232548 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.233345 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.233348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.233368 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:13.233669 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.233677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.235336 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.235341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.237478 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.237492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.239351 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.239357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.239520 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.239552 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.239648 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.239772 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15535,"mutex_wait_us":23,"thread_start_us":14481,"threads_started":1}
I20260811 06:20:13.240775 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.216951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.22.121.3:42009)
0811 06:20:13.230974 (+ 14023us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.230982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.232455 (+ 1473us) server_negotiation.cc:407] Connection header received
0811 06:20:13.232760 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.232767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.232973 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.233110 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:13.233833 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.235191 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.237625 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.238840 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.238954 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.239159 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.240323 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.240478 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13847,"thread_start_us":13560,"threads_started":1}
I20260811 06:20:13.242988 17025 tablet_service.cc:1467] Processing CreateTablet for tablet d5065bbd4fdb49389780d6890ae873c1 (DEFAULT_TABLE table=test-table [id=b8655fa84cdf48f5bcd7687477b5d289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:13.243367 17242 tablet_service.cc:1467] Processing CreateTablet for tablet d5065bbd4fdb49389780d6890ae873c1 (DEFAULT_TABLE table=test-table [id=b8655fa84cdf48f5bcd7687477b5d289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:13.244180 17025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5065bbd4fdb49389780d6890ae873c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:13.244151 17451 tablet_service.cc:1467] Processing CreateTablet for tablet d5065bbd4fdb49389780d6890ae873c1 (DEFAULT_TABLE table=test-table [id=b8655fa84cdf48f5bcd7687477b5d289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:13.244735 17242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5065bbd4fdb49389780d6890ae873c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:13.245405 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5065bbd4fdb49389780d6890ae873c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:13.255678 17593 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Bootstrap starting.
I20260811 06:20:13.259130 17593 tablet_bootstrap.cc:654] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:13.259179 17597 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Bootstrap starting.
I20260811 06:20:13.260604 17593 log.cc:824] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:13.260892 17599 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Bootstrap starting.
I20260811 06:20:13.262215 17597 tablet_bootstrap.cc:654] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:13.263376 17597 log.cc:824] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:13.263743 17599 tablet_bootstrap.cc:654] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:13.264209 17593 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: No bootstrap required, opened a new log
I20260811 06:20:13.264487 17593 ts_tablet_manager.cc:1397] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:20:13.264657 17599 log.cc:824] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:13.267505 17597 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: No bootstrap required, opened a new log
I20260811 06:20:13.267611 17599 tablet_bootstrap.cc:492] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: No bootstrap required, opened a new log
I20260811 06:20:13.267756 17597 ts_tablet_manager.cc:1397] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:20:13.267854 17599 ts_tablet_manager.cc:1397] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 06:20:13.274116 17597 raft_consensus.cc:348] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.274422 17597 raft_consensus.cc:374] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:13.274227 17593 raft_consensus.cc:348] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.274514 17597 raft_consensus.cc:729] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e46cf47ddffb46d39fda2cc9a36a4bf0, State: Initialized, Role: FOLLOWER
I20260811 06:20:13.274621 17593 raft_consensus.cc:374] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:13.274725 17593 raft_consensus.cc:729] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d61a09efd545f3af663e067ba0b945, State: Initialized, Role: FOLLOWER
I20260811 06:20:13.275112 17597 consensus_queue.cc:260] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.275380 17593 consensus_queue.cc:260] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.276206 17339 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:13.276827 17597 ts_tablet_manager.cc:1428] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 06:20:13.277551 17599 raft_consensus.cc:348] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.277962 17599 raft_consensus.cc:374] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:13.278079 17599 raft_consensus.cc:729] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f23e6cc87cc426ca8d3f1d15bf0263e, State: Initialized, Role: FOLLOWER
I20260811 06:20:13.278687 17599 consensus_queue.cc:260] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.280769 17560 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:13.281473 17599 ts_tablet_manager.cc:1428] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 06:20:13.284578 17135 heartbeater.cc:503] Master 127.22.121.62:41869 was elected leader, sending a full tablet report...
I20260811 06:20:13.285027 17593 ts_tablet_manager.cc:1428] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Time spent starting tablet: real 0.020s user 0.014s sys 0.003s
W20260811 06:20:13.301456 17136 tablet.cc:2369] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:20:13.406342 17562 tablet.cc:2369] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:20:13.485260 17340 tablet.cc:2369] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:20:13.523010 17616 raft_consensus.cc:482] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:20:13.523321 17616 raft_consensus.cc:504] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.525007 17616 leader_election.cc:288] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045), 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:13.531253 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.525361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:34625 (local address 127.22.121.2:40045)
0811 06:20:13.525610 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.525616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.525918 (+ 302us) server_negotiation.cc:407] Connection header received
0811 06:20:13.526234 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.526241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.526329 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.526445 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:13.527085 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.527878 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.529604 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.530382 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.530527 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.530601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.530885 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.531006 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:20:13.531338 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.525255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:40045 (local address 127.22.121.1:34625)
0811 06:20:13.525801 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.525817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.526044 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.526498 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.526502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.526528 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:13.526925 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.526954 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.528009 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.528014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.529445 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.529457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.530585 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.530591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.530734 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.530758 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.530840 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.530958 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":280,"threads_started":1}
I20260811 06:20:13.531759 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.526161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:33085 (local address 127.22.121.3:42009)
0811 06:20:13.526356 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.526362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.526775 (+ 413us) server_negotiation.cc:407] Connection header received
0811 06:20:13.526927 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.526931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.527000 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.527103 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:13.527675 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.528380 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.530055 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.530836 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.530980 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.531051 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.531387 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.531513 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:20:13.531850 17621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.525975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42009 (local address 127.22.121.1:33085)
0811 06:20:13.526669 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:13.526689 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:13.526811 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:13.527126 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:13.527130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:13.527154 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:13.527522 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.527532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.528511 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.528516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.529901 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:13.529913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.531050 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.531057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:13.531214 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.531238 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:20:13.531358 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 06:20:13.531476 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":3,"thread_start_us":348,"threads_started":1}
I20260811 06:20:13.532509 17274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5065bbd4fdb49389780d6890ae873c1" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" is_pre_election: true
I20260811 06:20:13.532850 17487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5065bbd4fdb49389780d6890ae873c1" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" is_pre_election: true
I20260811 06:20:13.532961 17274 raft_consensus.cc:2454] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 0.
I20260811 06:20:13.533306 17487 raft_consensus.cc:2454] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 0.
I20260811 06:20:13.533586 16947 leader_election.cc:302] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d61a09efd545f3af663e067ba0b945, e46cf47ddffb46d39fda2cc9a36a4bf0; no voters:
I20260811 06:20:13.533973 17616 raft_consensus.cc:2781] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:20:13.534118 17616 raft_consensus.cc:482] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:20:13.534227 17616 raft_consensus.cc:3037] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:13.536387 17616 raft_consensus.cc:504] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.536944 17616 leader_election.cc:288] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 election: Requested vote from peers e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045), 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009)
I20260811 06:20:13.537288 17274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5065bbd4fdb49389780d6890ae873c1" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
I20260811 06:20:13.537400 17487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5065bbd4fdb49389780d6890ae873c1" candidate_uuid: "09d61a09efd545f3af663e067ba0b945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
I20260811 06:20:13.537446 17274 raft_consensus.cc:3037] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:13.537585 17487 raft_consensus.cc:3037] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:13.539618 17274 raft_consensus.cc:2454] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 1.
I20260811 06:20:13.539746 17487 raft_consensus.cc:2454] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d61a09efd545f3af663e067ba0b945 in term 1.
I20260811 06:20:13.540004 16947 leader_election.cc:302] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d61a09efd545f3af663e067ba0b945, e46cf47ddffb46d39fda2cc9a36a4bf0; no voters:
I20260811 06:20:13.540304 17616 raft_consensus.cc:2781] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:13.540702 17616 raft_consensus.cc:686] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 1 LEADER]: Becoming Leader. State: Replica: 09d61a09efd545f3af663e067ba0b945, State: Running, Role: LEADER
I20260811 06:20:13.541137 17616 consensus_queue.cc:237] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } }
I20260811 06:20:13.545241 16803 catalog_manager.cc:5495] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d61a09efd545f3af663e067ba0b945 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "09d61a09efd545f3af663e067ba0b945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d61a09efd545f3af663e067ba0b945" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 } health_report { overall_health: UNKNOWN } } }
I20260811 06:20:13.588714 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.583466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.22.121.1:45441)
0811 06:20:13.583644 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.583650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.583715 (+ 65us) server_negotiation.cc:407] Connection header received
0811 06:20:13.583941 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.583945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.584027 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.584158 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:13.584538 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.585234 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.586600 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.586993 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.587111 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.587661 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:13.588141 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:20:13.588267 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.588379 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.588458 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:13.596423 17487 raft_consensus.cc:1261] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Refusing update from remote peer 09d61a09efd545f3af663e067ba0b945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:13.596514 17274 raft_consensus.cc:1261] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Refusing update from remote peer 09d61a09efd545f3af663e067ba0b945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:13.597357 17622 consensus_queue.cc:1035] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:13.597729 17616 consensus_queue.cc:1035] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 40045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:13.624361 16804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:13.624568 16804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:13.627848 16804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Sending DeleteTablet for 3 replicas of tablet d5065bbd4fdb49389780d6890ae873c1
I20260811 06:20:13.628813 17025 tablet_service.cc:1514] Processing DeleteTablet for tablet d5065bbd4fdb49389780d6890ae873c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:13 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:53920
I20260811 06:20:13.628806 17242 tablet_service.cc:1514] Processing DeleteTablet for tablet d5065bbd4fdb49389780d6890ae873c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:13 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:41520
I20260811 06:20:13.629669 17644 tablet_replica.cc:330] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: stopping tablet replica
I20260811 06:20:13.630100 17644 raft_consensus.cc:2229] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:13.630178 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet d5065bbd4fdb49389780d6890ae873c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:13 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:45716
I20260811 06:20:13.630635 17644 raft_consensus.cc:2258] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:13.631353 17645 tablet_replica.cc:330] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: stopping tablet replica
I20260811 06:20:13.631773 17645 raft_consensus.cc:2229] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:20:13.631947 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 16882
I20260811 06:20:13.632829 17644 ts_tablet_manager.cc:1905] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:13.633081 17645 raft_consensus.cc:2258] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:13.635296 17645 ts_tablet_manager.cc:1905] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:13.639604 17644 ts_tablet_manager.cc:1918] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:13.639827 17644 log.cc:1196] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/d5065bbd4fdb49389780d6890ae873c1
I20260811 06:20:13.640414 17644 ts_tablet_manager.cc:1939] T d5065bbd4fdb49389780d6890ae873c1 P e46cf47ddffb46d39fda2cc9a36a4bf0: Deleting consensus metadata
I20260811 06:20:13.642167 16784 catalog_manager.cc:4841] TS e46cf47ddffb46d39fda2cc9a36a4bf0 (127.22.121.2:40045): tablet d5065bbd4fdb49389780d6890ae873c1 (table test-table [id=b8655fa84cdf48f5bcd7687477b5d289]) successfully deleted
I20260811 06:20:13.642541 17645 ts_tablet_manager.cc:1918] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:13.642714 17645 log.cc:1196] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/d5065bbd4fdb49389780d6890ae873c1
I20260811 06:20:13.643312 17645 ts_tablet_manager.cc:1939] T d5065bbd4fdb49389780d6890ae873c1 P 09d61a09efd545f3af663e067ba0b945: Deleting consensus metadata
I20260811 06:20:13.644946 16786 catalog_manager.cc:4841] TS 09d61a09efd545f3af663e067ba0b945 (127.22.121.1:45441): tablet d5065bbd4fdb49389780d6890ae873c1 (table test-table [id=b8655fa84cdf48f5bcd7687477b5d289]) successfully deleted
I20260811 06:20:13.647840 17646 tablet_replica.cc:330] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: stopping tablet replica
I20260811 06:20:13.648269 17646 raft_consensus.cc:2229] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:13.648752 17646 raft_consensus.cc:2258] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:13.651036 17646 ts_tablet_manager.cc:1905] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:13.651487 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.633024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.22.121.1:45441)
0811 06:20:13.633288 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.633293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.633315 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:20:13.633365 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.633369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.633445 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.633548 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:13.635726 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.636926 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.638901 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.639567 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.639696 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.644821 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:13.644835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:13.644854 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:13.644900 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:13.648566 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.649415 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.649422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.649424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.649574 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.649975 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.649980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.649983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.650346 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:13.650690 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.650902 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.651104 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260811 06:20:13.652074 17122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:13.657260 17646 ts_tablet_manager.cc:1918] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:13.657400 17646 log.cc:1196] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/d5065bbd4fdb49389780d6890ae873c1
I20260811 06:20:13.657806 17646 ts_tablet_manager.cc:1939] T d5065bbd4fdb49389780d6890ae873c1 P 1f23e6cc87cc426ca8d3f1d15bf0263e: Deleting consensus metadata
I20260811 06:20:13.659111 16784 catalog_manager.cc:4841] TS 1f23e6cc87cc426ca8d3f1d15bf0263e (127.22.121.3:42009): tablet d5065bbd4fdb49389780d6890ae873c1 (table test-table [id=b8655fa84cdf48f5bcd7687477b5d289]) successfully deleted
I20260811 06:20:13.746256 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 16882
I20260811 06:20:13.770267 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 17142
I20260811 06:20:13.787735 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.770962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.22.121.2:40045)
0811 06:20:13.771137 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.771143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.771163 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:20:13.771364 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.771370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.771446 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.771531 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:13.773126 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.773893 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.775158 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.775721 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.775862 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.782287 (+ 6425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:13.782298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:13.782314 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:13.782347 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:13.784904 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.785603 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.785611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.785613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.785742 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.786141 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.786144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.786146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.786504 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:13.786770 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.787096 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.787331 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:20:13.788302 17331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:13.882237 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 17142
I20260811 06:20:13.909879 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 17344
I20260811 06:20:13.924536 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:13.910621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.22.121.3:42009)
0811 06:20:13.910791 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:13.910796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:13.910814 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:20:13.910940 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:13.910944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:13.911023 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:13.911117 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:13.912307 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.913027 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.914287 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:13.914654 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:13.914748 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:13.919651 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:13.919666 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:13.919684 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:13.919716 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:13.922174 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.922787 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.922795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.922797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.922926 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:13.923255 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:13.923258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:13.923275 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:13.923626 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:13.923840 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:13.924045 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:20:13.924212 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:20:13.925163 17552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:14.011579 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 17344
I20260811 06:20:14.037889 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 16752
I20260811 06:20:14.039052 16865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:14.153605 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 16752
I20260811 06:20:14.186957 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--webserver_interface=127.22.121.62
--webserver_port=32825
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:41869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:14.508823 17654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:14.509106 17654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:14.509248 17654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:14.516132 17654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:14.516238 17654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:14.516319 17654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:14.516402 17654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:14.516462 17654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:14.525647 17654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:41869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:41869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:14.526799 17654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:14.528056 17654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:14.538960 17665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:14.538916 17666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:14.539400 17676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:14.539945 17654 server_base.cc:1034] running on GCE node
I20260811 06:20:14.540647 17654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:14.541913 17654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:14.543119 17654 hybrid_clock.cc:648] HybridClock initialized: now 1786429214543059 us; error 43 us; skew 500 ppm
I20260811 06:20:14.543485 17654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:14 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429214, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:14.546293 17654 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:14.547633 17654 webserver.cc:460] Webserver started at http://127.22.121.62:32825/ using document root <none> and password file <none>
I20260811 06:20:14.548259 17654 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:14.548373 17654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:14.553467 17654 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 06:20:14.556100 17688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:14.557343 17654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:20:14.557507 17654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "8964a08bcacf45ff8b39419b05a4e226"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "6cdae3e57aba349c1b4ecbeaf4721300"
server_key_iv: "b8e7546bc60748eda3880700a290aada"
server_key_version: "encryptionkey@0"
I20260811 06:20:14.558399 17654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:14.584069 17654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:14.587497 17654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:14.587735 17654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:14.604336 17654 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:41869
I20260811 06:20:14.604370 17772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:41869 every 8 connection(s)
I20260811 06:20:14.605628 17654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:14.610608 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 17654
I20260811 06:20:14.611423 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:45441
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=39725
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:14.612167 17774 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:20:14.615283 17774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap starting.
I20260811 06:20:14.639724 17774 log.cc:824] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:14.660462 17774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:20:14.661165 17774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226: Bootstrap complete.
I20260811 06:20:14.674306 17774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:14.675506 17774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Initialized, Role: FOLLOWER
I20260811 06:20:14.676183 17774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:14.676453 17774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:14.676585 17774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:14.676761 17774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:20:14.682548 17774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:14.683169 17774 leader_election.cc:302] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8964a08bcacf45ff8b39419b05a4e226; no voters:
I20260811 06:20:14.683867 17774 leader_election.cc:288] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:20:14.683980 17785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:20:14.685938 17774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:14.690846 17785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [term 3 LEADER]: Becoming Leader. State: Replica: 8964a08bcacf45ff8b39419b05a4e226, State: Running, Role: LEADER
I20260811 06:20:14.691378 17785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } }
I20260811 06:20:14.693830 17786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:14.693822 17788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8964a08bcacf45ff8b39419b05a4e226. Latest consensus state: current_term: 3 leader_uuid: "8964a08bcacf45ff8b39419b05a4e226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8964a08bcacf45ff8b39419b05a4e226" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 41869 } } }
I20260811 06:20:14.694202 17786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:14.694200 17788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8964a08bcacf45ff8b39419b05a4e226 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:14.704772 17804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:14.712206 17804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:14.713725 17804 catalog_manager.cc:1259] Loaded cluster ID: 5b259b0c843341c793b10165b9e813e2
I20260811 06:20:14.713873 17804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 06:20:14.716409 17804 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc284dd99ad google::LogMessage::Fail() at ??:0
@ 0x7fc284ddc88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc284dd94de google::LogMessage::Flush() at ??:0
@ 0x7fc284ddaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc29a810512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc29a80d976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc29a895610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc29a895439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc287a25345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc287a2891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc287a28799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc287a03eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc28acdd6db start_thread at ??:0
@ 0x7fc283c7361f clone at ??:0
W20260811 06:20:15.048862 17780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:15.049144 17780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:15.049247 17780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:15.056155 17780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:15.056245 17780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:15.056356 17780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:20:15.065459 17780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:45441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=39725
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:15.066514 17780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:15.067745 17780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:15.079799 17850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.079846 17852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.079877 17854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:15.081161 17780 server_base.cc:1034] running on GCE node
I20260811 06:20:15.081688 17780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:15.082489 17780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:15.083676 17780 hybrid_clock.cc:648] HybridClock initialized: now 1786429215083629 us; error 38 us; skew 500 ppm
I20260811 06:20:15.083940 17780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:15 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:15.087046 17780 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:20:15.088421 17780 webserver.cc:460] Webserver started at http://127.22.121.1:39725/ using document root <none> and password file <none>
I20260811 06:20:15.089011 17780 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:15.089135 17780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:15.094046 17780 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 06:20:15.096581 17869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.097841 17780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:15.097973 17780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "09d61a09efd545f3af663e067ba0b945"
format_stamp: "Formatted at 2026-08-11 06:20:09 on dist-test-slave-gjrz"
server_key: "69b6738a6bf365bb3523299b5a494477"
server_key_iv: "c6cdd2d7eb8362b65c1bd4de86e173ed"
server_key_version: "encryptionkey@0"
I20260811 06:20:15.098850 17780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:15.138753 17780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:15.142747 17780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:15.142964 17780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:15.143889 17780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:15.145346 17780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:15.145437 17780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.145516 17780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:15.145570 17780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.176465 17780 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45441
I20260811 06:20:15.176532 18067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45441 every 8 connection(s)
I20260811 06:20:15.177819 17780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:20:15.182811 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.180803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:34007)
0811 06:20:15.181546 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.181779 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":258,"threads_started":1}
I20260811 06:20:15.183107 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 17780
W20260811 06:20:15.183795 18068 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:41869 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:41869: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
I20260811 06:20:15.183938 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:40045
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=37575
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:15.184739 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.184424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:58967)
0811 06:20:15.184558 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.184636 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 06:20:15.185607 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.185277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:38199)
0811 06:20:15.185394 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.185484 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:20:15.185815 18068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:20:15.505581 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:15.505808 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:15.505923 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:15.512823 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:15.512919 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:15.513068 18071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:20:15.522125 18071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:40045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=37575
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:15.523190 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:15.524417 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:15.536620 18080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.536744 18086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.536666 18089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:15.537791 18071 server_base.cc:1034] running on GCE node
I20260811 06:20:15.538348 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:15.539168 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:15.540375 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786429215540334 us; error 50 us; skew 500 ppm
I20260811 06:20:15.540666 18071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:15 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:15.543574 18071 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:20:15.544886 18071 webserver.cc:460] Webserver started at http://127.22.121.2:37575/ using document root <none> and password file <none>
I20260811 06:20:15.545523 18071 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:15.545642 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:15.550582 18071 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:15.553195 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.554488 18071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:20:15.554641 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e46cf47ddffb46d39fda2cc9a36a4bf0"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "3f6ef89cdbe50747307d82817f79cd20"
server_key_iv: "8a82b911ad10b2d373cf644dcbb69f76"
server_key_version: "encryptionkey@0"
I20260811 06:20:15.555635 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:15.571895 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:15.575325 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:15.575560 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:15.576434 18071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:15.577607 18071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:15.577713 18071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.577816 18071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:15.577888 18071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.608018 18071 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:40045
I20260811 06:20:15.608093 18314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:40045 every 8 connection(s)
I20260811 06:20:15.609390 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:20:15.613818 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.612134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:52809)
0811 06:20:15.612810 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.613016 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":255,"threads_started":1}
W20260811 06:20:15.614416 18315 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:41869 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:41869: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
I20260811 06:20:15.615128 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.614831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:37601)
0811 06:20:15.614954 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.615017 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:20:15.615875 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:15.615608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:50679)
0811 06:20:15.615706 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:15.615776 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 06:20:15.616083 18315 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:20:15.616612 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 18071
I20260811 06:20:15.617151 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:42009
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:41869
--webserver_port=34359
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:15.933014 18320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:15.933259 18320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:15.933391 18320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:15.940256 18320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:15.940362 18320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:15.940503 18320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:20:15.949661 18320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:42009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=34359
--tserver_master_addrs=127.22.121.62:41869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:15.950766 18320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:15.952003 18320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:15.964013 18332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.963963 18333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:15.964087 18335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:15.964929 18320 server_base.cc:1034] running on GCE node
I20260811 06:20:15.965469 18320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:15.966315 18320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:15.967491 18320 hybrid_clock.cc:648] HybridClock initialized: now 1786429215967437 us; error 46 us; skew 500 ppm
I20260811 06:20:15.967777 18320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:15 dist-test-slave-gjrz krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:15.970651 18320 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:20:15.971954 18320 webserver.cc:460] Webserver started at http://127.22.121.3:34359/ using document root <none> and password file <none>
I20260811 06:20:15.972565 18320 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:15.972690 18320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:15.977598 18320 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:20:15.980147 18346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:15.981349 18320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:15.981487 18320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1f23e6cc87cc426ca8d3f1d15bf0263e"
format_stamp: "Formatted at 2026-08-11 06:20:10 on dist-test-slave-gjrz"
server_key: "1b7f5ada8de63b690feb76b39884bdfd"
server_key_iv: "3a4dc29e48ef25089915ea5fb813d230"
server_key_version: "encryptionkey@0"
I20260811 06:20:15.982409 18320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:16.005419 18320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:16.009034 18320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:16.009276 18320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:16.010236 18320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:16.011533 18320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:16.011638 18320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:16.011744 18320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:16.011813 18320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:16.041875 18320 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42009
I20260811 06:20:16.041955 18553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42009 every 8 connection(s)
I20260811 06:20:16.043308 18320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:20:16.048019 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:16.046230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:52475)
0811 06:20:16.046936 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:16.047173 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":277,"threads_started":1}
W20260811 06:20:16.048623 18558 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:41869 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:41869: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
I20260811 06:20:16.049306 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:16.049006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:39287)
0811 06:20:16.049140 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:16.049203 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 06:20:16.049980 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:16.049722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:39625)
0811 06:20:16.049819 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:16.049885 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 06:20:16.050221 18558 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:20:16.050408 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 18320
I20260811 06:20:16.187047 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:16.186271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:53461)
0811 06:20:16.186727 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:16.186835 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260811 06:20:16.617266 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:16.616509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:50325)
0811 06:20:16.616917 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:16.617023 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":202,"threads_started":1}
I20260811 06:20:17.051429 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:17.050649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:39487)
0811 06:20:17.051079 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:17.051171 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":202,"threads_started":1}
I20260811 06:20:17.188587 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:17.187821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:60671)
0811 06:20:17.188313 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:17.188408 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":248,"threads_started":1}
I20260811 06:20:17.618657 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:17.618012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:45257)
0811 06:20:17.618427 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:17.618504 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":206,"threads_started":1}
I20260811 06:20:18.052875 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:18.052188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:34561)
0811 06:20:18.052637 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:18.052717 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":219,"threads_started":1}
I20260811 06:20:18.190034 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:18.189373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:41317)
0811 06:20:18.189791 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:18.189869 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":215,"threads_started":1}
I20260811 06:20:18.620030 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:18.619405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:39179)
0811 06:20:18.619803 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:18.619880 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":204,"threads_started":1}
I20260811 06:20:19.054276 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:19.053601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:50287)
0811 06:20:19.054002 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:19.054089 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":209,"threads_started":1}
I20260811 06:20:19.191483 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:19.190784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:45761)
0811 06:20:19.191204 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:19.191310 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":204,"threads_started":1}
I20260811 06:20:19.621426 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:19.620779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:35255)
0811 06:20:19.621167 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:19.621242 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":194,"threads_started":1}
I20260811 06:20:20.055739 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:20.055031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:38029)
0811 06:20:20.055470 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:20.055548 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":206,"threads_started":1}
I20260811 06:20:20.192931 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:20.192278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:56013)
0811 06:20:20.192693 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:20.192771 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":218,"threads_started":1}
I20260811 06:20:20.622886 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:20.622207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:52795)
0811 06:20:20.622652 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:20.622731 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":214,"threads_started":1}
I20260811 06:20:21.057138 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:21.056466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:37899)
0811 06:20:21.056893 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:21.056970 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":217,"threads_started":1}
I20260811 06:20:21.194365 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:21.193702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:48201)
0811 06:20:21.194120 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:21.194210 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260811 06:20:21.624431 18595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:21.623715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:52827)
0811 06:20:21.624191 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:21.624279 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":254,"threads_started":1}
I20260811 06:20:22.058637 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:22.057890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:50905)
0811 06:20:22.058348 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:22.058435 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":243,"threads_started":1}
I20260811 06:20:22.195955 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:22.195130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:54779)
0811 06:20:22.195663 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:22.195750 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":269,"threads_started":1}
I20260811 06:20:22.625926 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:22.625207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:51225)
0811 06:20:22.625647 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:22.625723 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":194,"threads_started":1}
I20260811 06:20:23.060221 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:23.059443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:49651)
0811 06:20:23.059929 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:23.060007 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":191,"threads_started":1}
I20260811 06:20:23.197333 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:23.196715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:60569)
0811 06:20:23.197103 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:23.197182 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":203,"threads_started":1}
I20260811 06:20:23.627318 18611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:23.626677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:59403)
0811 06:20:23.627064 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:23.627141 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":202,"threads_started":1}
I20260811 06:20:24.061614 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:24.060946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:50145)
0811 06:20:24.061362 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:24.061440 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":218,"threads_started":1}
I20260811 06:20:24.198824 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:24.198086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:48709)
0811 06:20:24.198551 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:24.198653 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260811 06:20:24.628732 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:24.628051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:56385)
0811 06:20:24.628487 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:24.628567 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":201,"threads_started":1}
I20260811 06:20:25.063048 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:25.062338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:48259)
0811 06:20:25.062805 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:25.062886 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":228,"threads_started":1}
I20260811 06:20:25.200327 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:25.199668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:60233)
0811 06:20:25.200097 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:25.200176 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":238,"threads_started":1}
I20260811 06:20:25.630167 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:25.629486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:49907)
0811 06:20:25.629906 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:25.629984 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":227,"threads_started":1}
I20260811 06:20:26.064474 18623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:26.063831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:45321)
0811 06:20:26.064254 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:26.064331 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":232,"threads_started":1}
I20260811 06:20:26.201767 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:26.201095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:59123)
0811 06:20:26.201503 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:26.201601 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":206,"threads_started":1}
I20260811 06:20:26.631699 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:26.630984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:54167)
0811 06:20:26.631428 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:26.631520 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":210,"threads_started":1}
I20260811 06:20:27.065865 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:27.065227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:51685)
0811 06:20:27.065616 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:27.065706 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":192,"threads_started":1}
I20260811 06:20:27.203186 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:27.202511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:34165)
0811 06:20:27.202959 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:27.203039 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":213,"threads_started":1}
I20260811 06:20:27.633147 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:27.632495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:56651)
0811 06:20:27.632909 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:27.632998 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":213,"threads_started":1}
I20260811 06:20:28.067371 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:28.066684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:38215)
0811 06:20:28.067102 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:28.067191 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":223,"threads_started":1}
I20260811 06:20:28.204665 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:28.203959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:37803)
0811 06:20:28.204403 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:28.204491 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260811 06:20:28.634655 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:28.633922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:43035)
0811 06:20:28.634386 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:28.634479 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260811 06:20:29.068929 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:29.068140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:50053)
0811 06:20:29.068639 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:29.068754 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":225,"threads_started":1}
I20260811 06:20:29.206044 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:29.205374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:43143)
0811 06:20:29.205795 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:29.205874 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260811 06:20:29.636157 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:29.635473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:42363)
0811 06:20:29.635887 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:29.635984 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":216,"threads_started":1}
I20260811 06:20:30.070346 18638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:30.069692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:43453)
0811 06:20:30.070094 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:30.070173 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":195,"threads_started":1}
I20260811 06:20:30.207676 18640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:30.206970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.1:57869)
0811 06:20:30.207405 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:30.207495 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":217,"threads_started":1}
I20260811 06:20:30.637830 18642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:30.637133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.2:33955)
0811 06:20:30.637554 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:30.637655 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":212,"threads_started":1}
I20260811 06:20:31.056792 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 17780
I20260811 06:20:31.072697 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:31.071471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:41869 (local address 127.22.121.3:49341)
0811 06:20:31.072414 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:31.072519 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":742,"threads_started":1}
I20260811 06:20:31.077773 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:31.057861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.22.121.1:45441)
0811 06:20:31.058475 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:31.058491 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:31.058533 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:20:31.060642 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:31.060652 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:31.061090 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:31.061375 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:31.062738 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.064067 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.065270 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.065913 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.066051 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:31.072307 (+ 6256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:31.072318 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:31.072337 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:31.072398 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:31.075048 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.075725 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.075731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.075737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.075899 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.076211 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.076215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.076217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.076683 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:31.076924 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:31.077215 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 06:20:31.077517 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":268,"threads_started":1}
I20260811 06:20:31.079113 18062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:31.161062 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 17780
I20260811 06:20:31.189606 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 18071
I20260811 06:20:31.206692 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:31.190472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.22.121.2:40045)
0811 06:20:31.190990 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:31.191004 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:31.191044 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:20:31.192687 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:31.192697 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:31.193135 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:31.193461 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:31.194941 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.196280 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.197516 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.198176 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.198352 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:31.200934 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:31.200942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:31.200964 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:31.201002 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:31.203606 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.204234 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.204240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.204246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.204413 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.204778 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.204782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.204784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.205272 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:31.205562 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:31.206036 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 06:20:31.206418 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260811 06:20:31.208060 18304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:31.289969 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 18071
I20260811 06:20:31.313669 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 18320
I20260811 06:20:31.331398 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:31.314571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.22.121.3:42009)
0811 06:20:31.315145 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:31.315160 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:31.315203 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:20:31.316866 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:31.316875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:31.317284 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:31.317626 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:31.319068 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.320428 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.321733 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:31.322455 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:31.322636 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:31.325276 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:31.325284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:31.325298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:31.325342 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:31.328199 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.328860 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.328868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.328874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.329045 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:31.329420 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:31.329424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:31.329426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:31.329933 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:31.330232 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:31.330684 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:20:31.331069 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":223,"threads_started":1}
I20260811 06:20:31.332751 18543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:31.414961 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 18320
2026-08-11T06:20:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24172 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:20:31 dist-test-slave-gjrz krb5kdc[18665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:20:31 dist-test-slave-gjrz krb5kdc[18665](info): set up 2 sockets
Aug 11 06:20:31 dist-test-slave-gjrz krb5kdc[18665](info): commencing operation
krb5kdc: starting...
W20260811 06:20:32.663659 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:20:32 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:20:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:20:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:20:32.888449 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:33.208506 18688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:33.208783 18688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:33.208927 18688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:33.215842 18688 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:20:33.215939 18688 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:20:33.216003 18688 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:20:33.216073 18688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:33.216148 18688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:33.216239 18688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:33.216298 18688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:33.216372 18688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:33.225759 18688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:33.226859 18688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:33.228116 18688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:33.239615 18698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:33.239565 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:33.239642 18696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:33.239854 18688 server_base.cc:1034] running on GCE node
I20260811 06:20:33.240778 18688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:33.242000 18688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:33.243244 18688 hybrid_clock.cc:648] HybridClock initialized: now 1786429233243177 us; error 33 us; skew 500 ppm
I20260811 06:20:33.243608 18688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:33 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429233, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:33.246670 18688 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:33.248071 18688 webserver.cc:460] Webserver started at http://127.22.121.62:42309/ using document root <none> and password file <none>
I20260811 06:20:33.248692 18688 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:33.248780 18688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:33.249073 18688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:33.251909 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "a405acda199647029d3296feaedeee10"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "b0134cbf0f6c4e8ac0d844afc1e272d2"
server_key_iv: "137cec37cfcb5f1b803ddaba22dd9142"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.252642 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "a405acda199647029d3296feaedeee10"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "b0134cbf0f6c4e8ac0d844afc1e272d2"
server_key_iv: "137cec37cfcb5f1b803ddaba22dd9142"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.257529 18688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:20:33.260720 18715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:33.261952 18688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:20:33.262108 18688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a405acda199647029d3296feaedeee10"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "b0134cbf0f6c4e8ac0d844afc1e272d2"
server_key_iv: "137cec37cfcb5f1b803ddaba22dd9142"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.262295 18688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:33.280454 18688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:33.284307 18688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:33.284533 18688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:33.301484 18688 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:33729
I20260811 06:20:33.301512 18802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:33729 every 8 connection(s)
I20260811 06:20:33.302675 18688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:33.306186 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 18688
I20260811 06:20:33.306425 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:20:33.306412 18804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:33.307312 23012 external_mini_cluster.cc:1428] Setting key 9a396695254664a0eaf26e85ebc858f8
I20260811 06:20:33.319737 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap starting.
Aug 11 06:20:33 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:33.323374 18804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:33.324518 18804 log.cc:824] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:33.327796 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: No bootstrap required, opened a new log
I20260811 06:20:33.332083 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:33.309309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.22.121.62:33729)
0811 06:20:33.310087 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:33.310097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:33.310138 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:20:33.312384 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:33.312406 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:33.312760 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:33.313156 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:33.314967 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.316308 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.318331 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.318944 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.319088 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:33.323956 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:33.323964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:33.323978 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:33.324012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:33.327756 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:33.328431 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:33.328440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:33.328448 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:33.328659 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:33.328998 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:33.329003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:33.329006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:33.329700 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:33.330003 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:33.330311 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 06:20:33.330743 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":354,"threads_started":1}
I20260811 06:20:33.337034 18804 raft_consensus.cc:348] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:33.337349 18804 raft_consensus.cc:374] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:33.337422 18804 raft_consensus.cc:729] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a405acda199647029d3296feaedeee10, State: Initialized, Role: FOLLOWER
I20260811 06:20:33.337920 18804 consensus_queue.cc:260] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:33.338128 18804 raft_consensus.cc:388] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:33.338227 18804 raft_consensus.cc:482] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:33.338343 18804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:33.340402 18804 raft_consensus.cc:504] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:33.340835 18804 leader_election.cc:302] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a405acda199647029d3296feaedeee10; no voters:
I20260811 06:20:33.341328 18804 leader_election.cc:288] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:20:33.341449 18811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:33.342139 18811 raft_consensus.cc:686] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 LEADER]: Becoming Leader. State: Replica: a405acda199647029d3296feaedeee10, State: Running, Role: LEADER
I20260811 06:20:33.342643 18811 consensus_queue.cc:237] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:33.343194 18804 sys_catalog.cc:564] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:33.345539 18812 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:20:33.345896 18812 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:33.346140 18813 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a405acda199647029d3296feaedeee10. Latest consensus state: current_term: 1 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:20:33.346532 18813 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:33.348186 18822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:33.352990 18822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:33.361773 18822 catalog_manager.cc:1347] Generated new cluster ID: 1c94f69fe13a439083d9380b23fd6976
I20260811 06:20:33.361866 18822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:33.383334 18822 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:20:33.385035 18822 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:33.396046 18822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Generated new TSK 0
I20260811 06:20:33.396747 18822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:20:33.456013 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:33.776875 18845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:33.777122 18845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:33.777210 18845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:33.784090 18845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:33.784181 18845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:33.784288 18845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:20:33.793367 18845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:33.794404 18845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:33.795586 18845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:33.807690 18852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:33.807721 18854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:33.807763 18857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:33.808485 18845 server_base.cc:1034] running on GCE node
I20260811 06:20:33.808991 18845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:33.809816 18845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:33.811009 18845 hybrid_clock.cc:648] HybridClock initialized: now 1786429233810971 us; error 33 us; skew 500 ppm
I20260811 06:20:33.811290 18845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:33 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429233, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:33.814242 18845 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:20:33.815621 18845 webserver.cc:460] Webserver started at http://127.22.121.1:36253/ using document root <none> and password file <none>
I20260811 06:20:33.816215 18845 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:33.816313 18845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:33.816556 18845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:33.819659 18845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "465fd7a234b5482abe663dca4e15d610"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "35c99cc169b5d30639a2865675b83f50"
server_key_iv: "586da8833c1190be650823210f1ab928"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.820338 18845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "465fd7a234b5482abe663dca4e15d610"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "35c99cc169b5d30639a2865675b83f50"
server_key_iv: "586da8833c1190be650823210f1ab928"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.825044 18845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:20:33.828280 18868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:33.829480 18845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:33.829618 18845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "465fd7a234b5482abe663dca4e15d610"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "35c99cc169b5d30639a2865675b83f50"
server_key_iv: "586da8833c1190be650823210f1ab928"
server_key_version: "encryptionkey@0"
I20260811 06:20:33.829779 18845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:33.861374 18845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:33.865252 18845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:33.865540 18845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:33.866417 18845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:33.867727 18845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:33.867799 18845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:33.867885 18845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:33.867941 18845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:33.897881 18845 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:38829
I20260811 06:20:33.898047 19064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:38829 every 8 connection(s)
I20260811 06:20:33.899091 18845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:20:33.908448 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 18845
I20260811 06:20:33.908635 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:20:33.909300 23012 external_mini_cluster.cc:1428] Setting key 1fe3b6eb439ff92c1388ac7c5f92157a
Aug 11 06:20:33 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429233, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:33.918606 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:33.901364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:59395 (local address 127.22.121.62:33729)
0811 06:20:33.901791 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:33.901796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:33.902728 (+ 932us) server_negotiation.cc:407] Connection header received
0811 06:20:33.904514 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:33.904520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:33.904596 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:33.904694 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:33.906123 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.906862 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.908429 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.908890 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.909011 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:33.912487 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:33.912494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:33.912499 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:33.912537 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:33.915469 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:33.916226 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:33.916231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:33.916234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:33.916378 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:33.916790 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:33.916795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:33.916798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:33.917117 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:33.917252 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:33.918035 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 06:20:33.918259 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":224,"threads_started":1}
I20260811 06:20:33.919351 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:33.901908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:59395)
0811 06:20:33.902567 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:33.902619 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:33.904257 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:33.904941 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:33.904949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:33.905419 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:33.905944 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:33.905961 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.907023 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.907027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:33.908283 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:33.908297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:33.909115 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:33.909123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:33.909356 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:33.909944 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:33.909966 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:33.912268 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:33.915596 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:33.915604 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:33.915618 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:33.916092 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:33.916471 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:33.916475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:33.916478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:33.916681 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:33.917222 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:33.917228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:33.917615 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 06:20:33.917877 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 06:20:33.918280 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":256,"threads_started":1}
I20260811 06:20:33.920732 19066 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:33.921025 19066 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:33.921797 19066 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:33.923998 18747 ts_manager.cc:194] Registered new tserver with Master: 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829)
I20260811 06:20:33.925410 18747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:59395
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:20:33.964818 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:34.285923 19078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:34.286264 19078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:34.286420 19078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:34.294597 19078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:34.294703 19078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:34.294863 19078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:20:34.306376 19078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:34.307816 19078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:34.309319 19078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:34.322921 19086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:34.322978 19091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:34.323005 19088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:34.323964 19078 server_base.cc:1034] running on GCE node
I20260811 06:20:34.324681 19078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:34.325793 19078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:34.327018 19078 hybrid_clock.cc:648] HybridClock initialized: now 1786429234326948 us; error 41 us; skew 500 ppm
I20260811 06:20:34.327379 19078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:34 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429234, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:34.331017 19078 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:20:34.333277 19078 webserver.cc:460] Webserver started at http://127.22.121.2:44581/ using document root <none> and password file <none>
I20260811 06:20:34.334080 19078 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:34.334192 19078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:34.334468 19078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:34.337234 19078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "45907d9eab6045459c11a3e59c1ef526"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "03cf9e1fc75d28ed256ee422c98d83c1"
server_key_iv: "dc2b07889896517fba16d7eca2e96081"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.337942 19078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "45907d9eab6045459c11a3e59c1ef526"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "03cf9e1fc75d28ed256ee422c98d83c1"
server_key_iv: "dc2b07889896517fba16d7eca2e96081"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.342634 19078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 06:20:34.345924 19103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.347126 19078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:20:34.347299 19078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "45907d9eab6045459c11a3e59c1ef526"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "03cf9e1fc75d28ed256ee422c98d83c1"
server_key_iv: "dc2b07889896517fba16d7eca2e96081"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.347472 19078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:34.382306 19078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:34.385813 19078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:34.386076 19078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:34.386947 19078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:34.388147 19078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:34.388227 19078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.388348 19078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:34.388398 19078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.417778 19078 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:43673
I20260811 06:20:34.417835 19292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:43673 every 8 connection(s)
I20260811 06:20:34.418929 19078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:20:34.428404 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 19078
I20260811 06:20:34.428601 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:20:34.429059 23012 external_mini_cluster.cc:1428] Setting key 29e5b435ed7702c70f44ce08e3a7a9eb
Aug 11 06:20:34 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429234, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:34.438180 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.421339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:56955 (local address 127.22.121.62:33729)
0811 06:20:34.421734 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.421739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.422605 (+ 866us) server_negotiation.cc:407] Connection header received
0811 06:20:34.424438 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.424444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.424526 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.424637 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:34.426101 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.426819 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.428710 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.429174 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.429275 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.432704 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:34.432711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:34.432716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:34.432751 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:34.435294 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.435976 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.435980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.435982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.436117 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.436531 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.436535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.436537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.436788 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:34.436925 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.437595 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.437812 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":196,"threads_started":1}
I20260811 06:20:34.439069 19304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.421753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:56955)
0811 06:20:34.422442 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:34.422479 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:34.424169 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:34.424881 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:34.424890 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:34.425386 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:34.425925 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.425942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.426972 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.426976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.428550 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.428567 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.429418 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.429427 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.429702 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.430334 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:34.430356 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:34.432478 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:34.435424 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:34.435433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:34.435449 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:34.435842 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:34.436222 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:34.436227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:34.436231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:34.436419 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:34.436908 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:34.436916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:34.437230 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 06:20:34.437463 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 06:20:34.437950 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":291,"threads_started":1}
I20260811 06:20:34.440394 19293 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:34.440670 19293 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:34.441349 19293 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:34.442713 18747 ts_manager.cc:194] Registered new tserver with Master: 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673)
I20260811 06:20:34.443583 18747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:56955
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:20:34.484145 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:34.801734 19311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:34.801995 19311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:34.802091 19311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:34.809036 19311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:34.809129 19311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:34.809240 19311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:20:34.818408 19311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:34.819545 19311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:34.820710 19311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:34.832890 19319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:34.833108 19321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:34.833335 19318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:34.833935 19311 server_base.cc:1034] running on GCE node
I20260811 06:20:34.834472 19311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:34.835335 19311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:34.836550 19311 hybrid_clock.cc:648] HybridClock initialized: now 1786429234836491 us; error 39 us; skew 500 ppm
I20260811 06:20:34.836834 19311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:34 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429234, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:34.839709 19311 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:20:34.840953 19311 webserver.cc:460] Webserver started at http://127.22.121.3:42865/ using document root <none> and password file <none>
I20260811 06:20:34.841513 19311 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:34.841624 19311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:34.841874 19311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:20:34.844655 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "990409698cbd4b438d45f6e4744ff1be"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "44ab28c1683ce0089dcdd614244a1ed9"
server_key_iv: "651627e9b5f88baa753ea560bf212493"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.845323 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "990409698cbd4b438d45f6e4744ff1be"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "44ab28c1683ce0089dcdd614244a1ed9"
server_key_iv: "651627e9b5f88baa753ea560bf212493"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.850111 19311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:20:34.853358 19334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.854537 19311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:20:34.854686 19311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "990409698cbd4b438d45f6e4744ff1be"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "44ab28c1683ce0089dcdd614244a1ed9"
server_key_iv: "651627e9b5f88baa753ea560bf212493"
server_key_version: "encryptionkey@0"
I20260811 06:20:34.854863 19311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:34.871941 19311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:34.875344 19311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:34.875589 19311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:34.876417 19311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:34.877619 19311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:34.877691 19311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.877800 19311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:34.877851 19311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:34.908169 19311 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42949
I20260811 06:20:34.908228 19553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42949 every 8 connection(s)
I20260811 06:20:34.909246 19311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:20:34.915251 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 19311
I20260811 06:20:34.915457 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:20:34.915867 23012 external_mini_cluster.cc:1428] Setting key 6e8102eb4216ca22b7e7fc3e0e6034f3
Aug 11 06:20:34 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429234, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:34.927286 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.911585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37329 (local address 127.22.121.62:33729)
0811 06:20:34.911757 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.911761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.912813 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 06:20:34.914541 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.914547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.914626 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.914721 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:34.916280 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.917040 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.918567 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.918954 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.919062 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.922154 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:34.922159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:34.922162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:34.922194 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:34.924450 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.925125 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.925129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.925131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.925247 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.925646 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.925650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.925652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.925929 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:34.926083 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.926722 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.926949 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:20:34.928090 19557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.911996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:37329)
0811 06:20:34.912670 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:34.912708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:34.914284 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:34.914979 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:34.914987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:34.915563 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:34.916118 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.916137 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.917196 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.917200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.918431 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.918445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.919168 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.919175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.919434 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.919979 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:34.920005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:34.921957 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:34.924586 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:34.924595 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:34.924609 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:34.924992 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:34.925351 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:34.925357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:34.925360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:34.925531 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:34.926055 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:34.926063 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:34.926368 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 06:20:34.926593 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 06:20:34.927036 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":250,"threads_started":1}
I20260811 06:20:34.928162 19066 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:34.929563 19554 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:34.929869 19554 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:34.930513 19554 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:34.931823 18745 ts_manager.cc:194] Registered new tserver with Master: 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949)
I20260811 06:20:34.932632 18745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:37329
I20260811 06:20:34.942627 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:20:34.958528 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.947080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.22.121.62:33729)
0811 06:20:34.947244 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.947250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.947755 (+ 505us) server_negotiation.cc:407] Connection header received
0811 06:20:34.947881 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.947884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.947954 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.948067 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:34.949461 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.950144 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.951558 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.951961 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.952049 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.954102 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:34.954107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:34.954110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:34.954137 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:34.956309 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.956898 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.956901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.956904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.957024 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:34.957313 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:34.957316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:34.957317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:34.957612 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:34.957817 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.958072 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.958223 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:34.962091 18745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:20:34.968760 18745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:20:34.990377 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.980646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.22.121.3:42949)
0811 06:20:34.981368 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.981377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.982166 (+ 789us) server_negotiation.cc:407] Connection header received
0811 06:20:34.982269 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.982276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.982509 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.982708 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:34.984109 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.985367 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.987845 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.989233 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.989342 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.989571 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.989869 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.990038 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":392,"threads_started":1}
I20260811 06:20:34.990423 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.980190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42949 (local address 127.0.0.1:42762)
0811 06:20:34.981746 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:34.981785 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:34.981983 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:34.982739 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:34.982746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:34.982776 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:34.983839 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.983855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.985478 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.985483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.987672 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.987690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.989406 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.989413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.989615 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.989674 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 06:20:34.989810 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:20:34.989983 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":21,"spinlock_wait_cycles":9088,"thread_start_us":1122,"threads_started":1}
I20260811 06:20:34.990517 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.980192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:38829 (local address 127.0.0.1:50356)
0811 06:20:34.981752 (+ 1560us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:34.981784 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:34.981983 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:34.983631 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:34.983635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:34.983655 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:34.984151 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.984165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.986118 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.986124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.988175 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.988186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.989726 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.989732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.989903 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.989934 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:20:34.990009 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 06:20:34.990136 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1460,"mutex_wait_us":189,"spinlock_wait_cycles":3200,"thread_start_us":1616,"threads_started":1}
I20260811 06:20:34.991747 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.982754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43673 (local address 127.0.0.1:58946)
0811 06:20:34.983491 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:34.983508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:34.983629 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:34.984884 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:34.984889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:34.984915 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:34.985233 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.985244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.986848 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.986854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.988972 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:34.988987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.990960 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.990967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:34.991148 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.991179 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:20:34.991290 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:20:34.991412 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":702,"threads_started":1}
I20260811 06:20:34.991747 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.980209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.22.121.1:38829)
0811 06:20:34.982596 (+ 2387us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.982607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.982633 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:20:34.982722 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.982729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.983426 (+ 697us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.984342 (+ 916us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:34.984551 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.985974 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.988310 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.989296 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.989421 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.989652 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.991292 (+ 1640us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.991449 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2223,"thread_start_us":2076,"threads_started":1}
I20260811 06:20:34.992439 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:34.983233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.22.121.2:43673)
0811 06:20:34.984325 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:34.984333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:34.984358 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:20:34.984449 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:34.984455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:34.984676 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:34.984873 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:34.985399 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.986701 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.989149 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:34.990761 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:34.990893 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:34.991086 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:34.991948 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 06:20:34.992126 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":774,"threads_started":1}
I20260811 06:20:34.994498 19429 tablet_service.cc:1467] Processing CreateTablet for tablet 7b02710009804792b83215733a35d421 (DEFAULT_TABLE table=test-table [id=835c22adedd14728bee3d2f2b26e6f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:34.994767 18964 tablet_service.cc:1467] Processing CreateTablet for tablet 7b02710009804792b83215733a35d421 (DEFAULT_TABLE table=test-table [id=835c22adedd14728bee3d2f2b26e6f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:34.996023 18964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b02710009804792b83215733a35d421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:34.996060 19429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b02710009804792b83215733a35d421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:34.996089 19182 tablet_service.cc:1467] Processing CreateTablet for tablet 7b02710009804792b83215733a35d421 (DEFAULT_TABLE table=test-table [id=835c22adedd14728bee3d2f2b26e6f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:34.997321 19182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b02710009804792b83215733a35d421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:35.006693 19586 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Bootstrap starting.
I20260811 06:20:35.010125 19586 tablet_bootstrap.cc:654] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:35.010301 19588 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Bootstrap starting.
I20260811 06:20:35.011582 19586 log.cc:824] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:35.011672 19590 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Bootstrap starting.
I20260811 06:20:35.013517 19588 tablet_bootstrap.cc:654] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:35.014528 19590 tablet_bootstrap.cc:654] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:35.014706 19588 log.cc:824] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:35.015057 19586 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: No bootstrap required, opened a new log
I20260811 06:20:35.015367 19586 ts_tablet_manager.cc:1397] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 06:20:35.015619 19590 log.cc:824] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:35.017693 19588 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: No bootstrap required, opened a new log
I20260811 06:20:35.017997 19588 ts_tablet_manager.cc:1397] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 06:20:35.018285 19590 tablet_bootstrap.cc:492] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: No bootstrap required, opened a new log
I20260811 06:20:35.018512 19590 ts_tablet_manager.cc:1397] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:20:35.024986 19590 raft_consensus.cc:348] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.025300 19590 raft_consensus.cc:374] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:35.025144 19586 raft_consensus.cc:348] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.025422 19590 raft_consensus.cc:729] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45907d9eab6045459c11a3e59c1ef526, State: Initialized, Role: FOLLOWER
I20260811 06:20:35.025527 19586 raft_consensus.cc:374] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:35.025635 19586 raft_consensus.cc:729] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 465fd7a234b5482abe663dca4e15d610, State: Initialized, Role: FOLLOWER
I20260811 06:20:35.026073 19590 consensus_queue.cc:260] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.026186 19586 consensus_queue.cc:260] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.027936 19586 ts_tablet_manager.cc:1428] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260811 06:20:35.028813 19588 raft_consensus.cc:348] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.029356 19588 raft_consensus.cc:374] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:35.029511 19588 raft_consensus.cc:729] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990409698cbd4b438d45f6e4744ff1be, State: Initialized, Role: FOLLOWER
I20260811 06:20:35.029783 19293 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:35.030396 19590 ts_tablet_manager.cc:1428] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260811 06:20:35.031006 19588 consensus_queue.cc:260] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.032157 19554 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:35.032655 19588 ts_tablet_manager.cc:1428] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
W20260811 06:20:35.151336 19067 tablet.cc:2369] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:20:35.160686 19555 tablet.cc:2369] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:20:35.170779 19294 tablet.cc:2369] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:20:35.181521 19595 raft_consensus.cc:482] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:20:35.181770 19595 raft_consensus.cc:504] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.183399 19595 leader_election.cc:288] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829), 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949)
I20260811 06:20:35.191826 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.183726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49315 (local address 127.22.121.1:38829)
0811 06:20:35.183970 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.183977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.184416 (+ 439us) server_negotiation.cc:407] Connection header received
0811 06:20:35.184627 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.184633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.184711 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.184798 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:35.185527 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.186650 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.189549 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.190748 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.190908 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.190981 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.191450 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.191588 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:20:35.191817 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.183800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:52995 (local address 127.22.121.3:42949)
0811 06:20:35.183986 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.183992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.184564 (+ 572us) server_negotiation.cc:407] Connection header received
0811 06:20:35.184714 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.184720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.184790 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.184868 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:35.185527 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.186645 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.189549 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.190749 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.190908 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.190981 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.191446 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.191578 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:20:35.192070 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.183608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42949 (local address 127.22.121.2:52995)
0811 06:20:35.184467 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:35.184482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:35.184590 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:35.184884 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:35.184888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:35.184906 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:35.185374 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:35.185388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.186767 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.186772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:35.189383 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:35.189400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.190976 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.190986 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:35.191239 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.191316 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 06:20:35.191405 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:20:35.191584 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":233,"thread_start_us":527,"threads_started":1}
I20260811 06:20:35.192085 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.183611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:38829 (local address 127.22.121.2:49315)
0811 06:20:35.184299 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:35.184322 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:35.184475 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:35.184833 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:35.184838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:35.184866 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:35.185389 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:35.185398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.186764 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.186769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:35.189383 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:35.189400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.190995 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.191000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:35.191237 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.191313 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 06:20:35.191402 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:20:35.191588 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":183,"thread_start_us":309,"threads_started":1}
I20260811 06:20:35.192953 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b02710009804792b83215733a35d421" candidate_uuid: "45907d9eab6045459c11a3e59c1ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "465fd7a234b5482abe663dca4e15d610" is_pre_election: true
I20260811 06:20:35.193069 19461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b02710009804792b83215733a35d421" candidate_uuid: "45907d9eab6045459c11a3e59c1ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990409698cbd4b438d45f6e4744ff1be" is_pre_election: true
I20260811 06:20:35.193437 18992 raft_consensus.cc:2454] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45907d9eab6045459c11a3e59c1ef526 in term 0.
I20260811 06:20:35.193485 19461 raft_consensus.cc:2454] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45907d9eab6045459c11a3e59c1ef526 in term 0.
I20260811 06:20:35.194056 19108 leader_election.cc:302] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45907d9eab6045459c11a3e59c1ef526, 465fd7a234b5482abe663dca4e15d610, 990409698cbd4b438d45f6e4744ff1be; no voters:
I20260811 06:20:35.194406 19595 raft_consensus.cc:2781] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:20:35.194526 19595 raft_consensus.cc:482] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:20:35.194624 19595 raft_consensus.cc:3037] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:35.196801 19595 raft_consensus.cc:504] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.197356 19595 leader_election.cc:288] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [CANDIDATE]: Term 1 election: Requested vote from peers 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829), 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949)
I20260811 06:20:35.197703 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b02710009804792b83215733a35d421" candidate_uuid: "45907d9eab6045459c11a3e59c1ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "465fd7a234b5482abe663dca4e15d610"
I20260811 06:20:35.197741 19461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b02710009804792b83215733a35d421" candidate_uuid: "45907d9eab6045459c11a3e59c1ef526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990409698cbd4b438d45f6e4744ff1be"
I20260811 06:20:35.197866 18992 raft_consensus.cc:3037] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:35.197896 19461 raft_consensus.cc:3037] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:35.200001 18992 raft_consensus.cc:2454] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45907d9eab6045459c11a3e59c1ef526 in term 1.
I20260811 06:20:35.200029 19461 raft_consensus.cc:2454] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45907d9eab6045459c11a3e59c1ef526 in term 1.
I20260811 06:20:35.200348 19108 leader_election.cc:302] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45907d9eab6045459c11a3e59c1ef526, 465fd7a234b5482abe663dca4e15d610; no voters:
I20260811 06:20:35.200632 19595 raft_consensus.cc:2781] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:35.201047 19595 raft_consensus.cc:686] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 1 LEADER]: Becoming Leader. State: Replica: 45907d9eab6045459c11a3e59c1ef526, State: Running, Role: LEADER
I20260811 06:20:35.201478 19595 consensus_queue.cc:237] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:35.205528 18745 catalog_manager.cc:5495] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2). New cstate: current_term: 1 leader_uuid: "45907d9eab6045459c11a3e59c1ef526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } health_report { overall_health: HEALTHY } } }
I20260811 06:20:35.222059 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.217080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.22.121.2:43673)
0811 06:20:35.217279 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.217283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.217299 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:20:35.217404 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.217407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.217485 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.217600 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:35.217891 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.218563 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.220336 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.220720 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.220810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.221162 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:35.221613 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:20:35.221715 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.221753 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.221812 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:20:35.229303 18992 raft_consensus.cc:1261] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Refusing update from remote peer 45907d9eab6045459c11a3e59c1ef526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:35.229310 19461 raft_consensus.cc:1261] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Refusing update from remote peer 45907d9eab6045459c11a3e59c1ef526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:35.230176 19603 consensus_queue.cc:1035] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:35.230477 19595 consensus_queue.cc:1035] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:35.240499 19610 mvcc.cc:205] Tried to move back new op lower bound from 7317214147489873920 to 7317214147388145664. Current Snapshot: MvccSnapshot[applied={T|T < 7317214147489873920}]
I20260811 06:20:35.242543 19612 mvcc.cc:205] Tried to move back new op lower bound from 7317214147489873920 to 7317214147388145664. Current Snapshot: MvccSnapshot[applied={T|T < 7317214147489873920}]
I20260811 06:20:35.253484 18747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:35.253700 18747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:35.257009 18747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Sending DeleteTablet for 3 replicas of tablet 7b02710009804792b83215733a35d421
I20260811 06:20:35.257999 18964 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b02710009804792b83215733a35d421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:35 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:50356
I20260811 06:20:35.258147 19429 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b02710009804792b83215733a35d421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:35 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:42762
I20260811 06:20:35.258435 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 18845
I20260811 06:20:35.259023 19182 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b02710009804792b83215733a35d421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:35 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:58946
I20260811 06:20:35.268071 19629 tablet_replica.cc:330] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: stopping tablet replica
I20260811 06:20:35.268553 19629 raft_consensus.cc:2229] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:20:35.268960 19626 tablet_replica.cc:330] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: stopping tablet replica
I20260811 06:20:35.269268 19629 raft_consensus.cc:2258] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:35.269812 19627 tablet_replica.cc:330] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: stopping tablet replica
I20260811 06:20:35.270110 19627 raft_consensus.cc:2229] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:35.270473 19627 raft_consensus.cc:2258] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:35.271646 19626 raft_consensus.cc:2229] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:35.272089 19626 raft_consensus.cc:2258] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:35.272503 19629 ts_tablet_manager.cc:1905] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:35.272785 19627 ts_tablet_manager.cc:1905] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:35.274147 19626 ts_tablet_manager.cc:1905] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:35.279477 19627 ts_tablet_manager.cc:1918] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:35.279668 19627 log.cc:1196] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/7b02710009804792b83215733a35d421
I20260811 06:20:35.280224 19627 ts_tablet_manager.cc:1939] T 7b02710009804792b83215733a35d421 P 990409698cbd4b438d45f6e4744ff1be: Deleting consensus metadata
Aug 11 06:20:35 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:20:35.281970 18717 catalog_manager.cc:4841] TS 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949): tablet 7b02710009804792b83215733a35d421 (table test-table [id=835c22adedd14728bee3d2f2b26e6f08]) successfully deleted
I20260811 06:20:35.283217 19626 ts_tablet_manager.cc:1918] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:35.283429 19626 log.cc:1196] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/7b02710009804792b83215733a35d421
I20260811 06:20:35.284027 19626 ts_tablet_manager.cc:1939] T 7b02710009804792b83215733a35d421 P 465fd7a234b5482abe663dca4e15d610: Deleting consensus metadata
I20260811 06:20:35.285763 18726 catalog_manager.cc:4841] TS 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829): tablet 7b02710009804792b83215733a35d421 (table test-table [id=835c22adedd14728bee3d2f2b26e6f08]) successfully deleted
I20260811 06:20:35.287926 19629 ts_tablet_manager.cc:1918] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:35.288172 19629 log.cc:1196] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/7b02710009804792b83215733a35d421
I20260811 06:20:35.288774 19629 ts_tablet_manager.cc:1939] T 7b02710009804792b83215733a35d421 P 45907d9eab6045459c11a3e59c1ef526: Deleting consensus metadata
I20260811 06:20:35.288897 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.259603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.22.121.1:38829)
0811 06:20:35.259771 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.259777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.267819 (+ 8042us) server_negotiation.cc:407] Connection header received
0811 06:20:35.269233 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.269239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.269326 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.269400 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:35.272458 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.273628 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.275645 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.276254 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.276353 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.282238 (+ 5885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:35.282252 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:35.282270 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:35.282315 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:35.286190 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.286852 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.286860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.286862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.286992 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.287395 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.287400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.287403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.287743 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:35.288013 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.288352 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.288550 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:20:35.289542 19058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:35.290614 18717 catalog_manager.cc:4841] TS 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673): tablet 7b02710009804792b83215733a35d421 (table test-table [id=835c22adedd14728bee3d2f2b26e6f08]) successfully deleted
I20260811 06:20:35.376616 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 18845
I20260811 06:20:35.400978 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 19078
Aug 11 06:20:35 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:20:35.416711 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.401726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.22.121.2:43673)
0811 06:20:35.401941 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.401950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.401971 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:20:35.402121 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.402127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.402199 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.402300 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:35.403605 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.404482 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.405788 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.406223 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.406341 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.410838 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:35.410854 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:35.410869 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:35.410907 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:35.413872 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.414513 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.414520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.414523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.414668 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.415031 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.415035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.415037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.415544 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:35.415871 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.416158 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.416378 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:20:35.417477 19286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:35.527686 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 19078
I20260811 06:20:35.549834 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 19311
Aug 11 06:20:35 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:20:35.563675 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:35.550504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.22.121.3:42949)
0811 06:20:35.550662 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:35.550666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:35.550734 (+ 68us) server_negotiation.cc:407] Connection header received
0811 06:20:35.550894 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:35.550898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:35.550973 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:35.551047 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:35.552332 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.553042 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.554171 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:35.554534 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:35.554624 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:35.558794 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:35.558809 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:35.558826 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:35.558861 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:35.561326 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.561970 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.561977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.561980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.562122 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:35.562400 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:35.562403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:35.562405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:35.562729 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:35.562942 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:35.563163 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 06:20:35.563431 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:20:35.564244 19540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:35.650064 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 19311
I20260811 06:20:35.672151 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 18688
I20260811 06:20:35.673210 18795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:35.779693 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 18688
I20260811 06:20:55.809183 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--webserver_interface=127.22.121.62
--webserver_port=42309
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:20:56.134182 19634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:56.134452 19634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:56.134583 19634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:56.141537 19634 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:20:56.141635 19634 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:20:56.141700 19634 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:20:56.141770 19634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:20:56.141840 19634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:56.141960 19634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:56.142021 19634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:20:56.142096 19634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:20:56.151711 19634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=42309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:56.152916 19634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:56.154218 19634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:56.177804 19649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:56.177832 19655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:56.177868 19652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:56.178479 19634 server_base.cc:1034] running on GCE node
I20260811 06:20:56.179327 19634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:56.180792 19634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:56.182025 19634 hybrid_clock.cc:648] HybridClock initialized: now 1786429256181948 us; error 49 us; skew 500 ppm
I20260811 06:20:56.182367 19634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:56 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429256, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:56.185815 19634 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:20:56.187378 19634 webserver.cc:460] Webserver started at http://127.22.121.62:42309/ using document root <none> and password file <none>
I20260811 06:20:56.188124 19634 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:56.188235 19634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:56.194053 19634 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 06:20:56.196982 19669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:56.198243 19634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:20:56.198402 19634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a405acda199647029d3296feaedeee10"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "b0134cbf0f6c4e8ac0d844afc1e272d2"
server_key_iv: "137cec37cfcb5f1b803ddaba22dd9142"
server_key_version: "encryptionkey@0"
I20260811 06:20:56.199450 19634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:56.213601 19634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:56.217576 19634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:56.217835 19634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:56.235466 19634 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:33729
I20260811 06:20:56.235518 19774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:33729 every 8 connection(s)
I20260811 06:20:56.236856 19634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:20:56.243904 19775 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:20:56.244872 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 19634
I20260811 06:20:56.245679 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:38829
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=36253
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:20:56.247411 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap starting.
I20260811 06:20:56.270258 19775 log.cc:824] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:56.286096 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:20:56.286679 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap complete.
I20260811 06:20:56.299499 19775 raft_consensus.cc:348] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:56.300622 19775 raft_consensus.cc:729] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a405acda199647029d3296feaedeee10, State: Initialized, Role: FOLLOWER
I20260811 06:20:56.301342 19775 consensus_queue.cc:260] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:56.301621 19775 raft_consensus.cc:388] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:20:56.301754 19775 raft_consensus.cc:482] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:20:56.301937 19775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:20:56.307490 19775 raft_consensus.cc:504] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:56.308113 19775 leader_election.cc:302] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a405acda199647029d3296feaedeee10; no voters:
I20260811 06:20:56.308748 19775 leader_election.cc:288] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:20:56.308871 19782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:20:56.310019 19782 raft_consensus.cc:686] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 LEADER]: Becoming Leader. State: Replica: a405acda199647029d3296feaedeee10, State: Running, Role: LEADER
I20260811 06:20:56.310564 19782 consensus_queue.cc:237] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:20:56.310947 19775 sys_catalog.cc:564] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:20:56.312990 19783 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:20:56.312983 19785 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a405acda199647029d3296feaedeee10. Latest consensus state: current_term: 2 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:20:56.313429 19783 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:56.313426 19785 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:20:56.320600 19793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:20:56.328356 19793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:20:56.329746 19793 catalog_manager.cc:1259] Loaded cluster ID: 1c94f69fe13a439083d9380b23fd6976
I20260811 06:20:56.329887 19793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:20:56.333830 19793 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:20:56.335390 19793 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Loaded TSK: 0
I20260811 06:20:56.354674 19814 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:20:56.372977 19793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Generated new TSK 1
I20260811 06:20:56.373366 19793 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:20:56.596827 19778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:56.597066 19778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:56.597179 19778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:56.604194 19778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:56.604286 19778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:56.604418 19778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:20:56.613660 19778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:38829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=36253
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:56.614758 19778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:56.615985 19778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:56.630030 19829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:56.630543 19833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:56.630579 19828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:56.631036 19778 server_base.cc:1034] running on GCE node
I20260811 06:20:56.631644 19778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:56.632536 19778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:56.633777 19778 hybrid_clock.cc:648] HybridClock initialized: now 1786429256633732 us; error 43 us; skew 500 ppm
I20260811 06:20:56.634065 19778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:56 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429256, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:56.637117 19778 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:20:56.638597 19778 webserver.cc:460] Webserver started at http://127.22.121.1:36253/ using document root <none> and password file <none>
I20260811 06:20:56.639199 19778 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:56.639343 19778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:56.644299 19778 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:20:56.646907 19850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:56.648212 19778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:56.648355 19778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "465fd7a234b5482abe663dca4e15d610"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "35c99cc169b5d30639a2865675b83f50"
server_key_iv: "586da8833c1190be650823210f1ab928"
server_key_version: "encryptionkey@0"
I20260811 06:20:56.649251 19778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:56.663669 19778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:56.667414 19778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:56.667654 19778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:56.668565 19778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:56.669859 19778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:56.669957 19778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:56.670054 19778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:56.670125 19778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:56.699841 19778 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:38829
I20260811 06:20:56.699908 20036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:38829 every 8 connection(s)
I20260811 06:20:56.701100 19778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:20:56.705564 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 19778
I20260811 06:20:56.706141 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:43673
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=44581
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:20:56 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429256, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:56.724148 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:56.703943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:34291)
0811 06:20:56.704685 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:56.704737 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:56.706488 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:56.707996 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:56.708002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:56.708457 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:56.708954 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:56.708992 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:56.711003 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:56.711008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:56.712294 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:56.712306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:56.713666 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:56.713670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:56.713849 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:56.714424 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:56.714445 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:56.716447 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:56.720111 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:56.720117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:56.720127 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:56.720494 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:56.720957 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:56.720962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:56.720974 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:56.721163 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:56.721904 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:56.721911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:56.722206 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 06:20:56.722455 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 06:20:56.722926 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":309,"threads_started":1}
I20260811 06:20:56.724160 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:56.703745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:34291 (local address 127.22.121.62:33729)
0811 06:20:56.704276 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:56.704287 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:56.704848 (+ 561us) server_negotiation.cc:407] Connection header received
0811 06:20:56.707059 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:56.707079 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:56.707526 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:56.707892 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:56.709174 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:56.710845 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:56.712442 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:56.713478 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:56.713649 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:56.716714 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:56.716722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:56.716735 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:56.716769 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:56.719963 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:56.720635 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:56.720643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:56.720650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:56.720842 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:56.721289 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:56.721293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:56.721295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:56.721764 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:56.721982 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:56.722684 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 06:20:56.723068 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260811 06:20:56.726912 20037 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:56.727236 20037 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:56.728103 20037 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:56.730312 19709 ts_manager.cc:194] Registered new tserver with Master: 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829)
I20260811 06:20:56.731796 19709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:34291
W20260811 06:20:57.025979 20042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:57.026207 20042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:57.026321 20042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:57.033560 20042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:57.033649 20042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:57.033789 20042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:20:57.042822 20042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:43673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=44581
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:57.043933 20042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:57.045125 20042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:57.057085 20053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:57.057178 20055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:57.057271 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:57.057772 20042 server_base.cc:1034] running on GCE node
I20260811 06:20:57.058311 20042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:57.059178 20042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:57.060370 20042 hybrid_clock.cc:648] HybridClock initialized: now 1786429257060324 us; error 40 us; skew 500 ppm
I20260811 06:20:57.060647 20042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:57 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429257, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:57.063634 20042 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:20:57.064965 20042 webserver.cc:460] Webserver started at http://127.22.121.2:44581/ using document root <none> and password file <none>
I20260811 06:20:57.065560 20042 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:57.065687 20042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:57.070752 20042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:20:57.073405 20068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.074597 20042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:20:57.074746 20042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "45907d9eab6045459c11a3e59c1ef526"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "03cf9e1fc75d28ed256ee422c98d83c1"
server_key_iv: "dc2b07889896517fba16d7eca2e96081"
server_key_version: "encryptionkey@0"
I20260811 06:20:57.075696 20042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:57.100484 20042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:57.104367 20042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:57.104602 20042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:57.105492 20042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:57.106732 20042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:57.106837 20042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.106941 20042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:57.107012 20042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.136641 20042 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:43673
I20260811 06:20:57.136715 20260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:43673 every 8 connection(s)
I20260811 06:20:57.137959 20042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:20:57.143870 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 20042
I20260811 06:20:57.144409 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:42949
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=42865
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:20:57 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429257, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:57.157404 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.140348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:53885 (local address 127.22.121.62:33729)
0811 06:20:57.140529 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.140534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.141605 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 06:20:57.143353 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.143357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.143424 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.143530 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.145056 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.146073 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.147684 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.148246 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.148367 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.151524 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.151530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.151534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.151573 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.154477 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.155274 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.155281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.155285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.155412 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.155838 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.155840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.155843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.156117 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.156223 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.156991 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.157174 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:20:57.158615 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.140759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:53885)
0811 06:20:57.141444 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.141491 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.143102 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.143699 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.143707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.144226 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:57.144882 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.144902 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.146264 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.146269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.147550 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.147562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.148425 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.148429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.148621 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.149202 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:57.149224 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:57.151255 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:57.154596 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:57.154602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:57.154613 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:57.155147 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:57.155509 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:57.155514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:57.155517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:57.155739 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:57.156240 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:57.156248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:57.156630 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.156903 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.157373 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":262,"threads_started":1}
I20260811 06:20:57.160038 20261 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:57.160308 20261 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:57.160933 20261 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:57.162252 19709 ts_manager.cc:194] Registered new tserver with Master: 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673)
I20260811 06:20:57.163028 19709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:53885
W20260811 06:20:57.464378 20265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:20:57.464635 20265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:20:57.464766 20265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:20:57.471655 20265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:20:57.471747 20265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:20:57.471897 20265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:20:57.481060 20265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:42949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=42865
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:20:57.482332 20265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:20:57.483690 20265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:20:57.496001 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:57.496168 20279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:20:57.496618 20273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:20:57.498708 20265 server_base.cc:1034] running on GCE node
I20260811 06:20:57.499425 20265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:20:57.500468 20265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:20:57.501660 20265 hybrid_clock.cc:648] HybridClock initialized: now 1786429257501606 us; error 50 us; skew 500 ppm
I20260811 06:20:57.501955 20265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:20:57 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429257, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:20:57.504918 20265 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:20:57.506203 20265 webserver.cc:460] Webserver started at http://127.22.121.3:42865/ using document root <none> and password file <none>
I20260811 06:20:57.506824 20265 fs_manager.cc:359] Metadata directory not provided
I20260811 06:20:57.506949 20265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:20:57.511925 20265 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:20:57.514447 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.515658 20265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:20:57.515802 20265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "990409698cbd4b438d45f6e4744ff1be"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "44ab28c1683ce0089dcdd614244a1ed9"
server_key_iv: "651627e9b5f88baa753ea560bf212493"
server_key_version: "encryptionkey@0"
I20260811 06:20:57.516680 20265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:20:57.529563 20265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:20:57.533039 20265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:20:57.533269 20265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:20:57.534134 20265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:20:57.535339 20265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:20:57.535431 20265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.535533 20265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:20:57.535605 20265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:20:57.565261 20265 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42949
I20260811 06:20:57.565331 20500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42949 every 8 connection(s)
I20260811 06:20:57.566571 20265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:20:57.571563 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 20265
Aug 11 06:20:57 dist-test-slave-gjrz krb5kdc[18665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429257, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:20:57.588274 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.572438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.22.121.62:33729)
0811 06:20:57.572843 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.572849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.573201 (+ 352us) server_negotiation.cc:407] Connection header received
0811 06:20:57.573330 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.573335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.573408 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.573528 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.575138 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.576109 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.578013 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.578535 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.578644 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.581525 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.581531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.581535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.581575 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.585429 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.586221 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.586226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.586229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.586418 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.586808 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.586811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.586813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.587215 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.587466 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.587758 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.587918 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":304,"spinlock_wait_cycles":3584,"thread_start_us":219,"threads_started":1}
I20260811 06:20:57.588668 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.568932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:44067 (local address 127.22.121.62:33729)
0811 06:20:57.569103 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.569107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.570216 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 06:20:57.572408 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.572414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.572494 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.572596 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.574509 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.575504 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.577528 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.577931 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.578078 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.581374 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.581379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.581382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.581414 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.585371 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.586228 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.586232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.586235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.586418 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.586878 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.586885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.586888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.587212 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.587374 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.588135 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.588335 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":896}
I20260811 06:20:57.589531 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.569367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:44067)
0811 06:20:57.570065 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.570101 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.572130 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.573090 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.573098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.573763 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:20:57.574324 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.574342 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.575658 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.575662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.577387 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.577402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.578149 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.578154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.578341 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.578990 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:20:57.579011 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:20:57.581181 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:20:57.585508 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:57.585526 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:57.585540 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:57.586131 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:57.586537 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:20:57.586542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:20:57.586546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:20:57.586776 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:20:57.587373 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:20:57.587380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:20:57.587763 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.588047 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.588470 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":260,"threads_started":1}
I20260811 06:20:57.590771 20501 heartbeater.cc:347] Connected to a master server at 127.22.121.62:33729
I20260811 06:20:57.591074 20501 heartbeater.cc:464] Registering TS with master...
I20260811 06:20:57.591760 20501 heartbeater.cc:511] Master 127.22.121.62:33729 requested a full tablet report, sending...
I20260811 06:20:57.593008 19709 ts_manager.cc:194] Registered new tserver with Master: 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949)
I20260811 06:20:57.593747 19709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:44067
I20260811 06:20:57.602955 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:20:57.609045 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.603774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.22.121.62:33729)
0811 06:20:57.603933 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.603939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.604327 (+ 388us) server_negotiation.cc:407] Connection header received
0811 06:20:57.604511 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.604516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.604637 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.604752 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:57.605189 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.605889 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.607243 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.607665 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.607770 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.608236 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:57.608474 (+ 238us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 06:20:57.608811 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48556: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 06:20:57.609138 20039 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48556: token expired
W20260811 06:20:57.609591 19559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.121.62:33729)
I20260811 06:20:57.621305 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.610036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.22.121.62:33729)
0811 06:20:57.610185 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.610189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.610203 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:20:57.610289 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.610292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.610339 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.610409 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.611740 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.612553 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.613922 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.614321 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.614406 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.616898 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.616903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.616906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.616932 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.619070 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.619692 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.619696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.619698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.619807 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.620140 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.620143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.620145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.620421 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.620638 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.620896 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.621062 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:20:57.624323 19709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:20:57.631206 19709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:20:57.661989 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.642611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.22.121.3:42949)
0811 06:20:57.652472 (+ 9861us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.652481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.653682 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 06:20:57.653794 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.653801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.654069 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.654262 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:57.655422 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.656673 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.659354 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.660765 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.660891 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.661127 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.661491 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.661652 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9159,"thread_start_us":8978,"threads_started":1}
I20260811 06:20:57.662012 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.641976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:42949 (local address 127.0.0.1:33558)
0811 06:20:57.653411 (+ 11435us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.653430 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.653554 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.654718 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.654727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.654758 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:57.655192 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.655204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.656813 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.656818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.659206 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.659218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.661006 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.661012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.661227 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.661281 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.661405 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.661585 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11311,"thread_start_us":522,"threads_started":1}
I20260811 06:20:57.665309 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.641556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:38829 (local address 127.0.0.1:48084)
0811 06:20:57.642674 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.642719 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.642868 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.658625 (+ 15757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.658630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.658659 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:57.659060 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.659072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.660807 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.660812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.662913 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.662926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.664567 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.664573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.664732 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.664755 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.664833 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.664947 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":60,"thread_start_us":15398,"threads_started":1}
I20260811 06:20:57.665365 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.641759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.22.121.1:38829)
0811 06:20:57.658077 (+ 16318us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.658085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.658111 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:20:57.658209 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.658216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.658420 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.658618 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:57.659302 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.660648 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.663063 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.664376 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.664505 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.664685 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.664894 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.665059 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16175,"thread_start_us":16063,"threads_started":1}
I20260811 06:20:57.665982 20378 tablet_service.cc:1467] Processing CreateTablet for tablet 739bd69b25f84972a5eb66139b71691f (DEFAULT_TABLE table=test-table [id=759187c0d2f7452396aa6a2f39a9af61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:57.666760 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.643197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43673 (local address 127.0.0.1:51576)
0811 06:20:57.657251 (+ 14054us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.657269 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.657389 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.658181 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.658186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.658212 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:57.658583 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.658597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.660606 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.660612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.664066 (+ 3454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.664082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.665997 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.666004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.666179 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.666210 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.666307 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.666421 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13907,"mutex_wait_us":87,"thread_start_us":11301,"threads_started":1}
I20260811 06:20:57.666795 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.643643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.22.121.2:43673)
0811 06:20:57.655843 (+ 12200us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.655850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.657514 (+ 1664us) server_negotiation.cc:407] Connection header received
0811 06:20:57.657613 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.657621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.657932 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.658177 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:57.658814 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.660434 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.664410 (+ 3976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.665778 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.665923 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.666104 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.666355 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.666508 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12036,"thread_start_us":12207,"threads_started":1}
I20260811 06:20:57.667886 20378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739bd69b25f84972a5eb66139b71691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:57.668934 19932 tablet_service.cc:1467] Processing CreateTablet for tablet 739bd69b25f84972a5eb66139b71691f (DEFAULT_TABLE table=test-table [id=759187c0d2f7452396aa6a2f39a9af61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:57.670248 19932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739bd69b25f84972a5eb66139b71691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:57.670996 20151 tablet_service.cc:1467] Processing CreateTablet for tablet 739bd69b25f84972a5eb66139b71691f (DEFAULT_TABLE table=test-table [id=759187c0d2f7452396aa6a2f39a9af61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:20:57.672255 20151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739bd69b25f84972a5eb66139b71691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:20:57.679208 20531 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Bootstrap starting.
I20260811 06:20:57.682634 20531 tablet_bootstrap.cc:654] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:57.683645 20531 log.cc:824] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:57.685870 20535 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Bootstrap starting.
I20260811 06:20:57.686368 20531 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: No bootstrap required, opened a new log
I20260811 06:20:57.686645 20531 ts_tablet_manager.cc:1397] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:20:57.687121 20536 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Bootstrap starting.
I20260811 06:20:57.689282 20535 tablet_bootstrap.cc:654] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:57.689930 20536 tablet_bootstrap.cc:654] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Neither blocks nor log segments found. Creating new log.
I20260811 06:20:57.690438 20535 log.cc:824] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:57.691066 20536 log.cc:824] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Log is configured to *not* fsync() on all Append() calls
I20260811 06:20:57.693166 20535 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: No bootstrap required, opened a new log
I20260811 06:20:57.693425 20535 ts_tablet_manager.cc:1397] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:20:57.694047 20536 tablet_bootstrap.cc:492] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: No bootstrap required, opened a new log
I20260811 06:20:57.694304 20536 ts_tablet_manager.cc:1397] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:20:57.695254 20531 raft_consensus.cc:348] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.695607 20531 raft_consensus.cc:374] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:57.695703 20531 raft_consensus.cc:729] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990409698cbd4b438d45f6e4744ff1be, State: Initialized, Role: FOLLOWER
I20260811 06:20:57.696230 20531 consensus_queue.cc:260] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.697176 20501 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:57.697794 20531 ts_tablet_manager.cc:1428] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 06:20:57.704219 20536 raft_consensus.cc:348] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.704522 20536 raft_consensus.cc:374] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:57.704607 20536 raft_consensus.cc:729] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45907d9eab6045459c11a3e59c1ef526, State: Initialized, Role: FOLLOWER
I20260811 06:20:57.705137 20536 consensus_queue.cc:260] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.705169 20535 raft_consensus.cc:348] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.705520 20535 raft_consensus.cc:374] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:20:57.705610 20535 raft_consensus.cc:729] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 465fd7a234b5482abe663dca4e15d610, State: Initialized, Role: FOLLOWER
I20260811 06:20:57.706171 20535 consensus_queue.cc:260] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.707770 20037 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:57.708490 20535 ts_tablet_manager.cc:1428] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 06:20:57.708914 20261 heartbeater.cc:503] Master 127.22.121.62:33729 was elected leader, sending a full tablet report...
I20260811 06:20:57.709941 20536 ts_tablet_manager.cc:1428] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260811 06:20:57.731612 20542 raft_consensus.cc:482] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:20:57.731917 20542 raft_consensus.cc:504] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.733379 20542 leader_election.cc:288] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829), 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673)
I20260811 06:20:57.738922 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.733922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:52345 (local address 127.22.121.2:43673)
0811 06:20:57.734129 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.734134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.734151 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:20:57.734203 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.734206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.734256 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.734328 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:57.734750 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.735736 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.737370 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.738120 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.738249 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.738318 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.738575 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.738694 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:20:57.738948 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.733658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43673 (local address 127.22.121.3:52345)
0811 06:20:57.733904 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.733920 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.734039 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.734365 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.734369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.734396 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:57.734604 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.734618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.735878 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.735882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.737216 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.737229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.738256 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.738262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.738427 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.738451 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.738539 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.738664 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 06:20:57.739568 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.733665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:37173 (local address 127.22.121.1:38829)
0811 06:20:57.733832 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.733838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.734529 (+ 691us) server_negotiation.cc:407] Connection header received
0811 06:20:57.734668 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.734673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.734742 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.734807 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:20:57.735411 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.736249 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.737883 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.738651 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.738785 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.738853 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.739175 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.739339 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:20:57.739612 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.733714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:38829 (local address 127.22.121.3:37173)
0811 06:20:57.734426 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 06:20:57.734444 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:20:57.734558 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:20:57.734845 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:20:57.734849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:20:57.734872 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:20:57.735239 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.735251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.736394 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.736398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.737737 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:20:57.737749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.738858 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.738866 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:20:57.739027 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.739063 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:20:57.739151 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:20:57.739292 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":420,"threads_started":1}
I20260811 06:20:57.739881 20189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739bd69b25f84972a5eb66139b71691f" candidate_uuid: "990409698cbd4b438d45f6e4744ff1be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45907d9eab6045459c11a3e59c1ef526" is_pre_election: true
I20260811 06:20:57.740288 20189 raft_consensus.cc:2454] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 990409698cbd4b438d45f6e4744ff1be in term 0.
I20260811 06:20:57.740386 19965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739bd69b25f84972a5eb66139b71691f" candidate_uuid: "990409698cbd4b438d45f6e4744ff1be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "465fd7a234b5482abe663dca4e15d610" is_pre_election: true
I20260811 06:20:57.740830 19965 raft_consensus.cc:2454] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 990409698cbd4b438d45f6e4744ff1be in term 0.
I20260811 06:20:57.740855 20298 leader_election.cc:302] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45907d9eab6045459c11a3e59c1ef526, 990409698cbd4b438d45f6e4744ff1be; no voters:
I20260811 06:20:57.741166 20542 raft_consensus.cc:2781] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:20:57.741286 20542 raft_consensus.cc:482] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:20:57.741390 20542 raft_consensus.cc:3037] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:57.743502 20542 raft_consensus.cc:504] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.744067 20542 leader_election.cc:288] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [CANDIDATE]: Term 1 election: Requested vote from peers 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829), 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673)
I20260811 06:20:57.744383 19965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739bd69b25f84972a5eb66139b71691f" candidate_uuid: "990409698cbd4b438d45f6e4744ff1be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "465fd7a234b5482abe663dca4e15d610"
I20260811 06:20:57.744513 20189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739bd69b25f84972a5eb66139b71691f" candidate_uuid: "990409698cbd4b438d45f6e4744ff1be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45907d9eab6045459c11a3e59c1ef526"
I20260811 06:20:57.744587 19965 raft_consensus.cc:3037] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:57.744693 20189 raft_consensus.cc:3037] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:20:57.747712 19965 raft_consensus.cc:2454] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 990409698cbd4b438d45f6e4744ff1be in term 1.
I20260811 06:20:57.747712 20189 raft_consensus.cc:2454] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 990409698cbd4b438d45f6e4744ff1be in term 1.
I20260811 06:20:57.748162 20303 leader_election.cc:302] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 465fd7a234b5482abe663dca4e15d610, 990409698cbd4b438d45f6e4744ff1be; no voters:
I20260811 06:20:57.748441 20542 raft_consensus.cc:2781] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:20:57.748843 20542 raft_consensus.cc:686] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 1 LEADER]: Becoming Leader. State: Replica: 990409698cbd4b438d45f6e4744ff1be, State: Running, Role: LEADER
I20260811 06:20:57.749264 20542 consensus_queue.cc:237] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } }
I20260811 06:20:57.753628 19709 catalog_manager.cc:5495] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be reported cstate change: term changed from 0 to 1, leader changed from <none> to 990409698cbd4b438d45f6e4744ff1be (127.22.121.3). New cstate: current_term: 1 leader_uuid: "990409698cbd4b438d45f6e4744ff1be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "990409698cbd4b438d45f6e4744ff1be" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 42949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 } health_report { overall_health: UNKNOWN } } }
I20260811 06:20:57.789237 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.784700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.22.121.3:42949)
0811 06:20:57.784857 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.784862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.784878 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:20:57.785054 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.785058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.785124 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.785235 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:20:57.785494 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.786179 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.787482 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.787867 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.787960 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.788348 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:20:57.788761 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:20:57.788861 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.788900 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.789011 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:20:57.796617 20189 raft_consensus.cc:1261] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Refusing update from remote peer 990409698cbd4b438d45f6e4744ff1be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:57.796618 19965 raft_consensus.cc:1261] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Refusing update from remote peer 990409698cbd4b438d45f6e4744ff1be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:20:57.797636 20542 consensus_queue.cc:1035] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [LEADER]: Connected to new peer: Peer: permanent_uuid: "45907d9eab6045459c11a3e59c1ef526" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:57.797922 20551 consensus_queue.cc:1035] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [LEADER]: Connected to new peer: Peer: permanent_uuid: "465fd7a234b5482abe663dca4e15d610" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:20:57.821183 19708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:57.821447 19708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:20:57.825157 19708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Sending DeleteTablet for 3 replicas of tablet 739bd69b25f84972a5eb66139b71691f
I20260811 06:20:57.826206 19932 tablet_service.cc:1514] Processing DeleteTablet for tablet 739bd69b25f84972a5eb66139b71691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:57 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:48084
I20260811 06:20:57.826373 20151 tablet_service.cc:1514] Processing DeleteTablet for tablet 739bd69b25f84972a5eb66139b71691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:57 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:51576
I20260811 06:20:57.826582 20378 tablet_service.cc:1514] Processing DeleteTablet for tablet 739bd69b25f84972a5eb66139b71691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:20:57 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:33558
I20260811 06:20:57.827692 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 19778
I20260811 06:20:57.837342 20572 tablet_replica.cc:330] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: stopping tablet replica
I20260811 06:20:57.837757 20572 raft_consensus.cc:2229] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:57.838215 20572 raft_consensus.cc:2258] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:57.839097 20574 tablet_replica.cc:330] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: stopping tablet replica
I20260811 06:20:57.839596 20574 raft_consensus.cc:2229] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:20:57.840214 20574 raft_consensus.cc:2258] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:57.840498 20572 ts_tablet_manager.cc:1905] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:57.842150 20574 ts_tablet_manager.cc:1905] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:57.845892 20573 tablet_replica.cc:330] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: stopping tablet replica
I20260811 06:20:57.847517 20572 ts_tablet_manager.cc:1918] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:57.847738 20572 log.cc:1196] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/739bd69b25f84972a5eb66139b71691f
I20260811 06:20:57.847970 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.828546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.22.121.1:38829)
0811 06:20:57.828694 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.828701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.828722 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:20:57.828866 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.828871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.828966 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.829069 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.830790 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.832040 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.833871 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.834559 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.834675 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.838865 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.838880 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.838895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.838939 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.842827 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.843713 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.843722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.843725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.843873 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.845069 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.845075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.845078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.845657 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.845971 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.847193 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.847445 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 06:20:57.848340 20572 ts_tablet_manager.cc:1939] T 739bd69b25f84972a5eb66139b71691f P 465fd7a234b5482abe663dca4e15d610: Deleting consensus metadata
I20260811 06:20:57.848824 20032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:57.849676 20574 ts_tablet_manager.cc:1918] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:57.849820 20573 raft_consensus.cc:2229] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:20:57.849897 20574 log.cc:1196] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/739bd69b25f84972a5eb66139b71691f
I20260811 06:20:57.850327 20573 raft_consensus.cc:2258] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:20:57.850895 20574 ts_tablet_manager.cc:1939] T 739bd69b25f84972a5eb66139b71691f P 990409698cbd4b438d45f6e4744ff1be: Deleting consensus metadata
I20260811 06:20:57.852443 19674 catalog_manager.cc:4841] TS 990409698cbd4b438d45f6e4744ff1be (127.22.121.3:42949): tablet 739bd69b25f84972a5eb66139b71691f (table test-table [id=759187c0d2f7452396aa6a2f39a9af61]) successfully deleted
I20260811 06:20:57.854089 20573 ts_tablet_manager.cc:1905] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:20:57.860790 20573 ts_tablet_manager.cc:1918] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:20:57.860937 20573 log.cc:1196] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/739bd69b25f84972a5eb66139b71691f
I20260811 06:20:57.861397 20573 ts_tablet_manager.cc:1939] T 739bd69b25f84972a5eb66139b71691f P 45907d9eab6045459c11a3e59c1ef526: Deleting consensus metadata
I20260811 06:20:57.862944 19674 catalog_manager.cc:4841] TS 45907d9eab6045459c11a3e59c1ef526 (127.22.121.2:43673): tablet 739bd69b25f84972a5eb66139b71691f (table test-table [id=759187c0d2f7452396aa6a2f39a9af61]) successfully deleted
I20260811 06:20:57.939126 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 19778
I20260811 06:20:57.939399 19677 catalog_manager.cc:4841] TS 465fd7a234b5482abe663dca4e15d610 (127.22.121.1:38829): tablet 739bd69b25f84972a5eb66139b71691f (table test-table [id=759187c0d2f7452396aa6a2f39a9af61]) successfully deleted
I20260811 06:20:57.965063 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 20042
I20260811 06:20:57.983469 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:57.965836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.22.121.2:43673)
0811 06:20:57.966030 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:57.966036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:57.966060 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:20:57.966155 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:57.966161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:57.966257 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:57.966364 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:57.967691 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.968793 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.970119 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:57.970779 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:57.970911 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:57.977967 (+ 7056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:57.977981 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:57.977997 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:57.978030 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:57.980851 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.981658 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.981665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.981667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.981789 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:57.982165 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:57.982169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:57.982171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:57.982500 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:57.982711 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:57.983008 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 06:20:57.983159 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:20:57.984113 20252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:58.070309 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 20042
I20260811 06:20:58.098114 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 20265
I20260811 06:20:58.114063 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:20:58.098902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.22.121.3:42949)
0811 06:20:58.099109 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:20:58.099115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:20:58.099139 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:20:58.099209 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:20:58.099215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:20:58.099331 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:20:58.099470 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:20:58.100760 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:58.102004 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:58.103576 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:20:58.104292 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:20:58.104440 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:20:58.106946 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:20:58.106965 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:20:58.106986 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:20:58.107040 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:20:58.111066 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:58.111836 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:58.111845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:58.111849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:58.112059 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:20:58.112451 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:20:58.112457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:20:58.112460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:20:58.112911 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:20:58.113229 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 06:20:58.113420 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 06:20:58.113640 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:20:58.114825 20495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:58.232452 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 20265
I20260811 06:20:58.260030 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 19634
I20260811 06:20:58.261109 19766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:20:58.370527 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 19634
I20260811 06:21:18.397984 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--webserver_interface=127.22.121.62
--webserver_port=42309
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:18.716902 20589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:18.717170 20589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:18.717294 20589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:18.724206 20589 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:21:18.724289 20589 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:21:18.724349 20589 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:21:18.724427 20589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:18.724493 20589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:18.724583 20589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:18.724642 20589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:18.724714 20589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:18.733979 20589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=42309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:18.735073 20589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:18.736344 20589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:18.747840 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:18.747875 20604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:18.747900 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:18.748555 20589 server_base.cc:1034] running on GCE node
I20260811 06:21:18.749284 20589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:18.750538 20589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:18.751750 20589 hybrid_clock.cc:648] HybridClock initialized: now 1786429278751691 us; error 30 us; skew 500 ppm
I20260811 06:21:18.752048 20589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:18 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:18.755023 20589 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:21:18.756417 20589 webserver.cc:460] Webserver started at http://127.22.121.62:42309/ using document root <none> and password file <none>
I20260811 06:21:18.757045 20589 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:18.757141 20589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:18.762251 20589 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 06:21:18.764930 20620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:18.766170 20589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:21:18.766319 20589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a405acda199647029d3296feaedeee10"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "b0134cbf0f6c4e8ac0d844afc1e272d2"
server_key_iv: "137cec37cfcb5f1b803ddaba22dd9142"
server_key_version: "encryptionkey@0"
I20260811 06:21:18.767207 20589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:18.793196 20589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:18.797049 20589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:18.797320 20589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:18.814432 20589 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:33729
I20260811 06:21:18.814465 20708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:33729 every 8 connection(s)
I20260811 06:21:18.815755 20589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:18.822758 20709 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:21:18.822760 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 20589
I20260811 06:21:18.823791 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:38829
--local_ip_for_outbound_sockets=127.22.121.1
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=36253
--webserver_interface=127.22.121.1
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:21:18.827073 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap starting.
I20260811 06:21:18.848511 20709 log.cc:824] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:18.863943 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:21:18.864413 20709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Bootstrap complete.
I20260811 06:21:18.873220 20709 raft_consensus.cc:348] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:21:18.874056 20709 raft_consensus.cc:729] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a405acda199647029d3296feaedeee10, State: Initialized, Role: FOLLOWER
I20260811 06:21:18.874643 20709 consensus_queue.cc:260] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:21:18.874866 20709 raft_consensus.cc:388] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:18.874974 20709 raft_consensus.cc:482] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:18.875164 20709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:21:18.878747 20709 raft_consensus.cc:504] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:21:18.879205 20709 leader_election.cc:302] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a405acda199647029d3296feaedeee10; no voters:
I20260811 06:21:18.879879 20709 leader_election.cc:288] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:21:18.880002 20717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:21:18.881099 20717 raft_consensus.cc:686] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [term 3 LEADER]: Becoming Leader. State: Replica: a405acda199647029d3296feaedeee10, State: Running, Role: LEADER
I20260811 06:21:18.881670 20717 consensus_queue.cc:237] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } }
I20260811 06:21:18.881923 20709 sys_catalog.cc:564] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:18.884027 20718 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:21:18.884393 20718 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:18.886610 20719 sys_catalog.cc:455] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a405acda199647029d3296feaedeee10. Latest consensus state: current_term: 3 leader_uuid: "a405acda199647029d3296feaedeee10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a405acda199647029d3296feaedeee10" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 33729 } } }
I20260811 06:21:18.886917 20719 sys_catalog.cc:458] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:18.888134 20725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:18.896989 20725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:18.898545 20725 catalog_manager.cc:1259] Loaded cluster ID: 1c94f69fe13a439083d9380b23fd6976
I20260811 06:21:18.898685 20725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:18.902890 20725 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:18.904592 20725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Loaded TSK: 0
I20260811 06:21:18.904774 20725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a405acda199647029d3296feaedeee10: Loaded TSK: 1
F20260811 06:21:18.906032 20725 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3c2244a9ad google::LogMessage::Fail() at ??:0
@ 0x7f3c2244d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3c2244a4de google::LogMessage::Flush() at ??:0
@ 0x7f3c2244ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3c261ae7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3c2619d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3c37e7d1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3c37e7c783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3c37f082a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3c37f080c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3c37f525e9 std::function<>::operator()() at ??:0
@ 0x7f3c37e80cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3c37e7ec36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3c37f06610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3c37f06439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c25096345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3c2509991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3c25099799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c25074eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3c2834e6db start_thread at ??:0
@ 0x7f3c212e461f clone at ??:0
W20260811 06:21:19.187495 20713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:19.187753 20713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:19.187888 20713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:19.194942 20713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:19.195048 20713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:19.195194 20713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:19.204694 20713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:38829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=36253
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:19.205840 20713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:19.207113 20713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:19.219580 20805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:19.219535 20804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:19.220058 20808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:19.220206 20713 server_base.cc:1034] running on GCE node
I20260811 06:21:19.220785 20713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:19.221614 20713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:19.222812 20713 hybrid_clock.cc:648] HybridClock initialized: now 1786429279222763 us; error 36 us; skew 500 ppm
I20260811 06:21:19.223097 20713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:19 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:19.226114 20713 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:21:19.227432 20713 webserver.cc:460] Webserver started at http://127.22.121.1:36253/ using document root <none> and password file <none>
I20260811 06:21:19.228031 20713 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:19.228133 20713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:19.233054 20713 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 06:21:19.235477 20819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.236661 20713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:21:19.236806 20713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "465fd7a234b5482abe663dca4e15d610"
format_stamp: "Formatted at 2026-08-11 06:20:33 on dist-test-slave-gjrz"
server_key: "35c99cc169b5d30639a2865675b83f50"
server_key_iv: "586da8833c1190be650823210f1ab928"
server_key_version: "encryptionkey@0"
I20260811 06:21:19.237731 20713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:19.274189 20713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:19.278231 20713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:19.278469 20713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:19.279414 20713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:19.280692 20713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:19.280788 20713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.280893 20713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:19.280977 20713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.311483 20713 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:38829
I20260811 06:21:19.311553 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:38829 every 8 connection(s)
I20260811 06:21:19.312815 20713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:19.313648 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 20713
I20260811 06:21:19.314426 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:43673
--local_ip_for_outbound_sockets=127.22.121.2
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=44581
--webserver_interface=127.22.121.2
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:21:19.317799 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.315973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:59487)
0811 06:21:19.316759 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.317008 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":294,"threads_started":1}
W20260811 06:21:19.318423 21031 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:33729 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:33729: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
I20260811 06:21:19.319178 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.318865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:54295)
0811 06:21:19.318984 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.319075 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 06:21:19.319972 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.319669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:33721)
0811 06:21:19.319794 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.319863 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 06:21:19.320194 21031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:21:19.636672 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:19.636922 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:19.637020 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:19.644058 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:19.644150 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:19.644268 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:19.653458 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:43673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=44581
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:19.654520 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:19.655735 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:19.667636 21043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:19.667812 21044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:19.667888 21050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:19.668967 21038 server_base.cc:1034] running on GCE node
I20260811 06:21:19.669497 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:19.670333 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:19.671522 21038 hybrid_clock.cc:648] HybridClock initialized: now 1786429279671481 us; error 44 us; skew 500 ppm
I20260811 06:21:19.671813 21038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:19 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:19.680013 21038 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:21:19.681284 21038 webserver.cc:460] Webserver started at http://127.22.121.2:44581/ using document root <none> and password file <none>
I20260811 06:21:19.681913 21038 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:19.682010 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:19.686926 21038 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:21:19.689399 21062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.690577 21038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:21:19.690721 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "45907d9eab6045459c11a3e59c1ef526"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "03cf9e1fc75d28ed256ee422c98d83c1"
server_key_iv: "dc2b07889896517fba16d7eca2e96081"
server_key_version: "encryptionkey@0"
I20260811 06:21:19.691638 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:19.704195 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:19.707957 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:19.708199 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:19.709111 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:19.710287 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:19.710379 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.710453 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:19.710506 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:19.740399 21038 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:43673
I20260811 06:21:19.740464 21272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:43673 every 8 connection(s)
I20260811 06:21:19.741791 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:19.746280 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.744482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:48933)
0811 06:21:19.745220 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.745443 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":260,"threads_started":1}
W20260811 06:21:19.746852 21275 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:33729 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:33729: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
I20260811 06:21:19.747596 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.747316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:52187)
0811 06:21:19.747428 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.747494 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 06:21:19.748265 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:19.748000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:56481)
0811 06:21:19.748100 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:19.748168 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 06:21:19.748466 21275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:21:19.748951 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 21038
I20260811 06:21:19.749536 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:42949
--local_ip_for_outbound_sockets=127.22.121.3
--tserver_master_addrs=127.22.121.62:33729
--webserver_port=42865
--webserver_interface=127.22.121.3
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:20.060372 21281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:20.060613 21281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:20.060729 21281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:20.067595 21281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:20.067682 21281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:20.067812 21281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:20.076754 21281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:42949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=42865
--tserver_master_addrs=127.22.121.62:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:20.077843 21281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:20.079036 21281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:20.091030 21289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:20.091056 21288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:20.091459 21293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:20.091813 21281 server_base.cc:1034] running on GCE node
I20260811 06:21:20.092367 21281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:20.093206 21281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:20.094393 21281 hybrid_clock.cc:648] HybridClock initialized: now 1786429280094350 us; error 38 us; skew 500 ppm
I20260811 06:21:20.094674 21281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:20 dist-test-slave-gjrz krb5kdc[18665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429280, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:20.097522 21281 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:21:20.098866 21281 webserver.cc:460] Webserver started at http://127.22.121.3:42865/ using document root <none> and password file <none>
I20260811 06:21:20.099522 21281 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:20.099639 21281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:20.104512 21281 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 06:21:20.107071 21306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:20.108335 21281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:21:20.108481 21281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "990409698cbd4b438d45f6e4744ff1be"
format_stamp: "Formatted at 2026-08-11 06:20:34 on dist-test-slave-gjrz"
server_key: "44ab28c1683ce0089dcdd614244a1ed9"
server_key_iv: "651627e9b5f88baa753ea560bf212493"
server_key_version: "encryptionkey@0"
I20260811 06:21:20.109447 21281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:20.128307 21281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:20.131691 21281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:20.131920 21281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:20.132771 21281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:20.133956 21281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:20.134052 21281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:20.134163 21281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:20.134240 21281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:20.164693 21281 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:42949
I20260811 06:21:20.164750 21503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:42949 every 8 connection(s)
I20260811 06:21:20.166003 21281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:20.170473 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:20.168727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:45289)
0811 06:21:20.169450 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:20.169676 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":300,"threads_started":1}
W20260811 06:21:20.171068 21505 heartbeater.cc:644] Failed to heartbeat to 127.22.121.62:33729 (0 consecutive failures): Network error: Failed to ping master at 127.22.121.62:33729: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
I20260811 06:21:20.171769 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:20.171481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:35765)
0811 06:21:20.171603 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:20.171673 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 06:21:20.172461 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:20.172201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:51999)
0811 06:21:20.172298 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:20.172364 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 06:21:20.172657 21505 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:21:20.173345 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 21281
I20260811 06:21:20.321416 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:20.320646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:55167)
0811 06:21:20.321093 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:20.321199 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":207,"threads_started":1}
I20260811 06:21:20.749709 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:20.748928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:37585)
0811 06:21:20.749382 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:20.749494 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260811 06:21:21.173997 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:21.173154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:58661)
0811 06:21:21.173642 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:21.173754 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":203,"threads_started":1}
I20260811 06:21:21.322854 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:21.322180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:49597)
0811 06:21:21.322620 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:21.322696 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260811 06:21:21.751107 21518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:21.750485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:59155)
0811 06:21:21.750885 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:21.750959 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":189,"threads_started":1}
I20260811 06:21:22.175482 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:22.174795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:48179)
0811 06:21:22.175209 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:22.175320 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":201,"threads_started":1}
I20260811 06:21:22.324302 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:22.323642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:49719)
0811 06:21:22.324054 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:22.324143 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":227,"threads_started":1}
I20260811 06:21:22.752447 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:22.751826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:51939)
0811 06:21:22.752216 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:22.752292 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
I20260811 06:21:23.176885 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:23.176233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:32969)
0811 06:21:23.176647 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:23.176724 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
I20260811 06:21:23.325891 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:23.325189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:56149)
0811 06:21:23.325654 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:23.325730 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":203,"threads_started":1}
I20260811 06:21:23.753855 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:23.753181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:52761)
0811 06:21:23.753592 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:23.753672 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":201,"threads_started":1}
I20260811 06:21:24.178362 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:24.177660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:45359)
0811 06:21:24.178099 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:24.178178 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":217,"threads_started":1}
I20260811 06:21:24.327441 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:24.326742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:48001)
0811 06:21:24.327169 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:24.327245 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":226,"threads_started":1}
I20260811 06:21:24.755357 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:24.754641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:58475)
0811 06:21:24.755070 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:24.755159 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260811 06:21:25.179867 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:25.179154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:52469)
0811 06:21:25.179621 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:25.179697 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":254,"threads_started":1}
I20260811 06:21:25.328919 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:25.328204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:46999)
0811 06:21:25.328641 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:25.328729 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":228,"threads_started":1}
I20260811 06:21:25.756776 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:25.756105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:52757)
0811 06:21:25.756526 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:25.756616 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260811 06:21:26.181334 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:26.180646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:54733)
0811 06:21:26.181087 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:26.181179 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":227,"threads_started":1}
I20260811 06:21:26.330446 21546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:26.329684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:55481)
0811 06:21:26.330164 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:26.330243 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":197,"threads_started":1}
I20260811 06:21:26.758374 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:26.757537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:47425)
0811 06:21:26.758077 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:26.758159 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":248,"threads_started":1}
I20260811 06:21:27.182938 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:27.182156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:36877)
0811 06:21:27.182654 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:27.182733 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":219,"threads_started":1}
I20260811 06:21:27.331925 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:27.331252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:48363)
0811 06:21:27.331697 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:27.331775 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260811 06:21:27.759979 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:27.759192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:45411)
0811 06:21:27.759671 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:27.759752 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":246,"threads_started":1}
I20260811 06:21:28.184429 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:28.183772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:52373)
0811 06:21:28.184198 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:28.184274 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260811 06:21:28.333438 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:28.332762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:51341)
0811 06:21:28.333194 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:28.333278 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":203,"threads_started":1}
I20260811 06:21:28.761492 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:28.760807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:32951)
0811 06:21:28.761235 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:28.761314 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":214,"threads_started":1}
I20260811 06:21:29.185873 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:29.185194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:48563)
0811 06:21:29.185631 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:29.185710 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":207,"threads_started":1}
I20260811 06:21:29.334899 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:29.334234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:59179)
0811 06:21:29.334664 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:29.334741 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260811 06:21:29.763060 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:29.762342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:52293)
0811 06:21:29.762771 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:29.762870 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":240,"threads_started":1}
I20260811 06:21:30.187383 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:30.186667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:42637)
0811 06:21:30.187119 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:30.187201 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
I20260811 06:21:30.336400 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:30.335733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:58133)
0811 06:21:30.336143 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:30.336230 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":206,"threads_started":1}
I20260811 06:21:30.764618 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:30.763878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:41971)
0811 06:21:30.764370 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:30.764448 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":265,"threads_started":1}
I20260811 06:21:31.188824 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:31.188150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:53881)
0811 06:21:31.188566 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:31.188655 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":197,"threads_started":1}
I20260811 06:21:31.337885 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:31.337242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:37443)
0811 06:21:31.337663 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:31.337740 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":203,"threads_started":1}
I20260811 06:21:31.766101 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:31.765454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:59521)
0811 06:21:31.765872 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:31.765951 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":217,"threads_started":1}
I20260811 06:21:32.190222 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:32.189612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:40893)
0811 06:21:32.189999 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:32.190076 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":195,"threads_started":1}
I20260811 06:21:32.339390 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:32.338661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:59247)
0811 06:21:32.339091 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:32.339182 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":199,"threads_started":1}
I20260811 06:21:32.767697 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:32.766937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:53423)
0811 06:21:32.767421 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:32.767524 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":217,"threads_started":1}
I20260811 06:21:33.191823 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:33.191036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:58015)
0811 06:21:33.191552 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:33.191646 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":280,"threads_started":1}
I20260811 06:21:33.340893 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:33.340161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:34177)
0811 06:21:33.340579 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:33.340665 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":207,"threads_started":1}
I20260811 06:21:33.769200 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:33.768524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:35091)
0811 06:21:33.768945 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:33.769029 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":198,"threads_started":1}
I20260811 06:21:34.193257 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:34.192603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:38141)
0811 06:21:34.192997 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:34.193087 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":201,"threads_started":1}
I20260811 06:21:34.342546 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:34.341821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.1:42991)
0811 06:21:34.342281 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:34.342366 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":272,"threads_started":1}
I20260811 06:21:34.770859 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:34.770139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.2:34223)
0811 06:21:34.770585 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:34.770683 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":230,"threads_started":1}
I20260811 06:21:35.181614 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 20713
I20260811 06:21:35.194952 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:35.194263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:33729 (local address 127.22.121.3:37711)
0811 06:21:35.194690 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:35.194777 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.62:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":228,"threads_started":1}
I20260811 06:21:35.199061 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:35.182750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54090 (local address 127.22.121.1:38829)
0811 06:21:35.183384 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:35.183397 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:35.183435 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:21:35.185007 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:35.185017 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:35.185437 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:35.185770 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:35.187245 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.188486 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.189749 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.190399 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.190567 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:35.193062 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:35.193070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:35.193084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:35.193118 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:35.196106 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.196738 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.196744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.196750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.196914 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.197270 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.197274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.197276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.197734 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:35.198020 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:35.198419 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 06:21:35.198790 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":323,"threads_started":1}
I20260811 06:21:35.200390 21020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:35.281471 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 20713
I20260811 06:21:35.307255 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 21038
I20260811 06:21:35.324200 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:35.308184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.22.121.2:43673)
0811 06:21:35.308686 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:35.308700 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:35.308737 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:21:35.310324 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:35.310333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:35.310752 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:35.311072 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:35.312604 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.313889 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.315148 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.315847 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.316016 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:35.318438 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:35.318446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:35.318461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:35.318502 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:35.321070 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.321715 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.321721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.321728 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.321911 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.322270 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.322273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.322275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.322767 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:35.323056 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:35.323508 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:21:35.323895 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260811 06:21:35.325537 21265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:35.406890 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 21038
I20260811 06:21:35.432830 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 21281
I20260811 06:21:35.449687 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:35.434035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.22.121.3:42949)
0811 06:21:35.434649 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:35.434663 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:35.434706 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:21:35.436392 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:35.436402 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:35.436840 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:35.437117 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:35.438391 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.439735 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.440905 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:35.441558 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:35.441705 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:35.444336 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:35.444344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:35.444358 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:35.444393 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:35.446972 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.447631 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.447638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.447644 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.447812 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:35.448080 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:35.448083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:35.448085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:35.448544 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:35.448790 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:35.449068 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 06:21:35.449373 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":288,"threads_started":1}
I20260811 06:21:35.450954 21498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:35.537546 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 21281
2026-08-11T06:21:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64121 ms)
[----------] 30 tests from SecurityITest (295070 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:21:35 dist-test-slave-gjrz krb5kdc[21613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:21:35 dist-test-slave-gjrz krb5kdc[21613](info): set up 2 sockets
Aug 11 06:21:35 dist-test-slave-gjrz krb5kdc[21613](info): commencing operation
krb5kdc: starting...
W20260811 06:21:36.773221 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:21:36 dist-test-slave-gjrz krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:21:37.004277 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46625
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:37.321395 21633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:37.321653 21633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:37.321746 21633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:37.328719 21633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:37.328812 21633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:37.328861 21633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:37.328915 21633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:37.328956 21633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:37.338419 21633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:37.339599 21633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:37.340844 21633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:37.351755 21641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:37.351848 21642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:37.352036 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:37.352905 21633 server_base.cc:1034] running on GCE node
I20260811 06:21:37.353556 21633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:37.354769 21633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:37.355958 21633 hybrid_clock.cc:648] HybridClock initialized: now 1786429297355921 us; error 25 us; skew 500 ppm
I20260811 06:21:37.356278 21633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:37 dist-test-slave-gjrz krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:37.359105 21633 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:21:37.360507 21633 webserver.cc:460] Webserver started at http://127.22.121.62:38475/ using document root <none> and password file <none>
I20260811 06:21:37.361136 21633 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:37.361251 21633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:37.361529 21633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:37.364352 21633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "11ed029ed91c46cc851e7749e3a23d86"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "07a8f104142ed78ed54f8f73ecbc6959"
server_key_iv: "60338011dc406bb943e4609816cefe52"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.365074 21633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "11ed029ed91c46cc851e7749e3a23d86"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "07a8f104142ed78ed54f8f73ecbc6959"
server_key_iv: "60338011dc406bb943e4609816cefe52"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.369778 21633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:21:37.372921 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:37.374080 21633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:21:37.374231 21633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "11ed029ed91c46cc851e7749e3a23d86"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "07a8f104142ed78ed54f8f73ecbc6959"
server_key_iv: "60338011dc406bb943e4609816cefe52"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.374389 21633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:37.394351 21633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:37.397956 21633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:37.398201 21633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:37.414860 21633 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46625
I20260811 06:21:37.414886 21768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46625 every 8 connection(s)
I20260811 06:21:37.416056 21633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:37.419440 21769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:37.426523 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 21633
I20260811 06:21:37.426770 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:37.427603 23012 external_mini_cluster.cc:1428] Setting key 2d82db2e3e04fda4ff65a559c6964373
I20260811 06:21:37.430102 21769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: Bootstrap starting.
I20260811 06:21:37.433234 21769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:37.434496 21769 log.cc:824] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:37.437474 21769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: No bootstrap required, opened a new log
Aug 11 06:21:37 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:37.445945 21769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } }
I20260811 06:21:37.446305 21769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:37.446414 21769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11ed029ed91c46cc851e7749e3a23d86, State: Initialized, Role: FOLLOWER
I20260811 06:21:37.447054 21769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } }
I20260811 06:21:37.447361 21769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:37.447506 21769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:37.447697 21769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:37.450091 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:37.429583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.22.121.62:46625)
0811 06:21:37.430345 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:37.430356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:37.430393 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:21:37.432102 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:37.432122 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:37.432458 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:37.432756 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:37.434603 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:37.435959 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:37.437830 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:37.438491 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:37.438610 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:37.443282 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:37.443292 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:37.443313 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:37.443346 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:37.446179 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:37.446973 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:37.446979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:37.446985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:37.447144 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:37.447595 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:37.447600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:37.447603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:37.448118 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:37.448346 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:37.448798 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:21:37.449127 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":343,"threads_started":1}
I20260811 06:21:37.450171 21769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } }
I20260811 06:21:37.450692 21769 leader_election.cc:302] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11ed029ed91c46cc851e7749e3a23d86; no voters:
I20260811 06:21:37.451218 21769 leader_election.cc:288] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:37.451386 21776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:37.452541 21776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [term 1 LEADER]: Becoming Leader. State: Replica: 11ed029ed91c46cc851e7749e3a23d86, State: Running, Role: LEADER
I20260811 06:21:37.453146 21776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } }
I20260811 06:21:37.453354 21769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:37.455856 21780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11ed029ed91c46cc851e7749e3a23d86. Latest consensus state: current_term: 1 leader_uuid: "11ed029ed91c46cc851e7749e3a23d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } } }
I20260811 06:21:37.456228 21780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:37.456631 21779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11ed029ed91c46cc851e7749e3a23d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11ed029ed91c46cc851e7749e3a23d86" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46625 } } }
I20260811 06:21:37.456919 21779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:37.459558 21794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:37.464761 21794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:37.472707 21794 catalog_manager.cc:1347] Generated new cluster ID: 5b1087c79b274d229c87d551307f7e4c
I20260811 06:21:37.472796 21794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:37.483685 21794 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:37.484838 21794 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:37.499790 21794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: Generated new TSK 0
I20260811 06:21:37.500488 21794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:21:37.572111 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:37.915946 21820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:37.916188 21820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:37.916285 21820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:37.923038 21820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:37.923123 21820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:37.923233 21820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:37.932186 21820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:37.933218 21820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:37.934346 21820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:37.946270 21833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:37.946342 21830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:37.946473 21836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:37.947222 21820 server_base.cc:1034] running on GCE node
I20260811 06:21:37.947844 21820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:37.948694 21820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:37.949911 21820 hybrid_clock.cc:648] HybridClock initialized: now 1786429297949862 us; error 34 us; skew 500 ppm
I20260811 06:21:37.950202 21820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:37 dist-test-slave-gjrz krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:37.953070 21820 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:21:37.954414 21820 webserver.cc:460] Webserver started at http://127.22.121.1:44777/ using document root <none> and password file <none>
I20260811 06:21:37.955082 21820 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:37.955195 21820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:37.955492 21820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:37.958220 21820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "69305f6a89b9430c846530554a44b605"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "9845af934e67af7bc8aaa4205d76a8fb"
server_key_iv: "93c6fbca4d7ff0623453b2a71aabf375"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.958938 21820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "69305f6a89b9430c846530554a44b605"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "9845af934e67af7bc8aaa4205d76a8fb"
server_key_iv: "93c6fbca4d7ff0623453b2a71aabf375"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.963721 21820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:21:37.966852 21848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:37.968072 21820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:37.968214 21820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "69305f6a89b9430c846530554a44b605"
format_stamp: "Formatted at 2026-08-11 06:21:37 on dist-test-slave-gjrz"
server_key: "9845af934e67af7bc8aaa4205d76a8fb"
server_key_iv: "93c6fbca4d7ff0623453b2a71aabf375"
server_key_version: "encryptionkey@0"
I20260811 06:21:37.968376 21820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:37.990859 21820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:37.994243 21820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:37.994486 21820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:37.995402 21820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:37.996585 21820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:37.996665 21820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:37.996779 21820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:37.996831 21820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.026067 21820 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:46607
I20260811 06:21:38.026118 22093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:46607 every 8 connection(s)
I20260811 06:21:38.027259 21820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:38.036051 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 21820
I20260811 06:21:38.036266 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:38.037075 23012 external_mini_cluster.cc:1428] Setting key b26f85b9644d8551e2808e0a775c82d1
Aug 11 06:21:38 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:38.046892 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:38.029633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:60703 (local address 127.22.121.62:46625)
0811 06:21:38.030025 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:38.030029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:38.030980 (+ 951us) server_negotiation.cc:407] Connection header received
0811 06:21:38.032734 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:38.032739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:38.032817 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:38.032911 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:38.034291 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.035012 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.036973 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.037519 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.037652 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:38.041282 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:38.041288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:38.041291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:38.041327 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:38.044156 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:38.044821 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:38.044826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:38.044829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:38.044987 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:38.045345 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:38.045348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:38.045351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:38.045650 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:38.045797 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:38.046356 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 06:21:38.046558 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":192,"threads_started":1}
I20260811 06:21:38.047566 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:38.030167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46625 (local address 127.22.121.1:60703)
0811 06:21:38.030822 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:38.030865 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:38.032507 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:38.033092 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:38.033100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:38.033580 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:38.034114 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:38.034131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.035141 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.035145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:38.036825 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:38.036841 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.037752 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.037760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:38.038064 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:38.038731 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:38.038759 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:38.041064 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:38.044294 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:38.044301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:38.044311 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:38.044702 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:38.045078 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:38.045081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:38.045084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:38.045244 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:38.045745 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:38.045750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:38.046017 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 06:21:38.046219 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 06:21:38.046547 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":241,"threads_started":1}
I20260811 06:21:38.048933 22094 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46625
I20260811 06:21:38.049197 22094 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:38.049814 22094 heartbeater.cc:511] Master 127.22.121.62:46625 requested a full tablet report, sending...
I20260811 06:21:38.051749 21707 ts_manager.cc:194] Registered new tserver with Master: 69305f6a89b9430c846530554a44b605 (127.22.121.1:46607)
I20260811 06:21:38.053167 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:60703
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:21:38.094245 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:38.404383 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:38.404624 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:38.404719 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:38.411461 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:38.411548 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:38.411656 22101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:38.420517 22101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:38.421572 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:38.422772 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:38.434736 22111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:38.434711 22113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:38.434893 22115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:38.435442 22101 server_base.cc:1034] running on GCE node
I20260811 06:21:38.435940 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:38.436692 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:38.437870 22101 hybrid_clock.cc:648] HybridClock initialized: now 1786429298437834 us; error 30 us; skew 500 ppm
I20260811 06:21:38.438149 22101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:38 dist-test-slave-gjrz krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:38.441128 22101 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:21:38.442430 22101 webserver.cc:460] Webserver started at http://127.22.121.2:35361/ using document root <none> and password file <none>
I20260811 06:21:38.443032 22101 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:38.443147 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:38.443471 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:38.446084 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "6ffdcd112512435db65e9ae018c3c46f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "1110123f3c0f39fc9a5c4f3b4e80af5c"
server_key_iv: "30f15254abaf6ee31f10f306f2edbb2d"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.446786 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "6ffdcd112512435db65e9ae018c3c46f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "1110123f3c0f39fc9a5c4f3b4e80af5c"
server_key_iv: "30f15254abaf6ee31f10f306f2edbb2d"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.451519 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 06:21:38.454770 22130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.456001 22101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:21:38.456146 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "6ffdcd112512435db65e9ae018c3c46f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "1110123f3c0f39fc9a5c4f3b4e80af5c"
server_key_iv: "30f15254abaf6ee31f10f306f2edbb2d"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.456315 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:38.489884 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:38.493723 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:38.493969 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:38.494824 22101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:38.496050 22101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:38.496129 22101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.496250 22101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:38.496304 22101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.525354 22101 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:45845
I20260811 06:21:38.525409 22333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:45845 every 8 connection(s)
I20260811 06:21:38.526474 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:38.533811 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 22101
I20260811 06:21:38.533998 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:38.534412 23012 external_mini_cluster.cc:1428] Setting key 3b3a3815162513d6b076651164aa8576
Aug 11 06:21:38 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:38.545125 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:38.528749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55097 (local address 127.22.121.62:46625)
0811 06:21:38.528903 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:38.528908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:38.530065 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 06:21:38.531876 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:38.531882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:38.531963 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:38.532051 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:38.533445 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.534210 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.535891 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.536444 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.536557 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:38.539794 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:38.539799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:38.539802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:38.539834 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:38.542289 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:38.542935 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:38.542941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:38.542945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:38.543070 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:38.543513 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:38.543518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:38.543520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:38.543797 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:38.543930 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:38.544567 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 06:21:38.544803 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:21:38.545900 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:38.529242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46625 (local address 127.22.121.2:55097)
0811 06:21:38.529914 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:38.529952 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:38.531623 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:38.532290 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:38.532298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:38.532759 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:38.533276 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:38.533293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.534344 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.534348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:38.535751 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:38.535763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:38.536662 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:38.536671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:38.536882 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:38.537471 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:38.537496 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:38.539588 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:38.542413 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:38.542421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:38.542436 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:38.542812 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:38.543173 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:38.543178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:38.543182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:38.543401 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:38.543920 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:38.543928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:38.544235 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 06:21:38.544449 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 06:21:38.544897 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":275,"threads_started":1}
I20260811 06:21:38.547169 22335 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46625
I20260811 06:21:38.547443 22335 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:38.548115 22335 heartbeater.cc:511] Master 127.22.121.62:46625 requested a full tablet report, sending...
I20260811 06:21:38.549495 21707 ts_manager.cc:194] Registered new tserver with Master: 6ffdcd112512435db65e9ae018c3c46f (127.22.121.2:45845)
I20260811 06:21:38.550271 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:55097
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:21:38.590370 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:38.899374 22349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:38.899614 22349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:38.899712 22349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:38.906582 22349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:38.906667 22349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:38.906777 22349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:38.915877 22349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:38.916908 22349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:38.918077 22349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:38.930034 22360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:38.930064 22363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:38.930476 22359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:38.930954 22349 server_base.cc:1034] running on GCE node
I20260811 06:21:38.931540 22349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:38.932365 22349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:38.933540 22349 hybrid_clock.cc:648] HybridClock initialized: now 1786429298933494 us; error 46 us; skew 500 ppm
I20260811 06:21:38.933820 22349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:38 dist-test-slave-gjrz krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:38.936708 22349 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:21:38.937914 22349 webserver.cc:460] Webserver started at http://127.22.121.3:35967/ using document root <none> and password file <none>
I20260811 06:21:38.938493 22349 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:38.938589 22349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:38.938832 22349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:38.941588 22349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "bfbf31ff82754bcba1c61061f37fc82f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "b684d7dec7bbfc8dc398adecf1771995"
server_key_iv: "8f741beddac0f4a48a40fb5b56da1e73"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.942281 22349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "bfbf31ff82754bcba1c61061f37fc82f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "b684d7dec7bbfc8dc398adecf1771995"
server_key_iv: "8f741beddac0f4a48a40fb5b56da1e73"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.946947 22349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 06:21:38.950078 22373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.951236 22349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:21:38.951421 22349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "bfbf31ff82754bcba1c61061f37fc82f"
format_stamp: "Formatted at 2026-08-11 06:21:38 on dist-test-slave-gjrz"
server_key: "b684d7dec7bbfc8dc398adecf1771995"
server_key_iv: "8f741beddac0f4a48a40fb5b56da1e73"
server_key_version: "encryptionkey@0"
I20260811 06:21:38.951594 22349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:38.965785 22349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:38.969172 22349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:38.969408 22349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:38.970278 22349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:38.971540 22349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:38.971621 22349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:38.971735 22349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:38.971791 22349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:39.001170 22349 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:45973
I20260811 06:21:39.001228 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:45973 every 8 connection(s)
I20260811 06:21:39.002362 22349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:39.010965 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 22349
I20260811 06:21:39.011124 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:39.011620 23012 external_mini_cluster.cc:1428] Setting key 9caefdf4ed91d6a7e9b287c6db5d33bf
Aug 11 06:21:39 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:39.020419 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.004699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:58255 (local address 127.22.121.62:46625)
0811 06:21:39.004902 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.004907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.006037 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 06:21:39.007772 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.007776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.007859 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.007944 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:39.009257 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.009998 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.011589 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.012049 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.012164 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.015700 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:39.015704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:39.015708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:39.015738 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:39.018001 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.018618 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.018622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.018624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.018749 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.019078 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.019081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.019083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.019353 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:39.019476 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.020002 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.020173 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:21:39.021286 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.005192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46625 (local address 127.22.121.3:58255)
0811 06:21:39.005881 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.005919 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.007541 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.008138 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.008145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.008609 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:39.009130 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.009147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.010127 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.010131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.011449 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.011464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.012272 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.012280 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.012524 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.013139 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:39.013160 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:39.015524 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:39.018128 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:39.018135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:39.018145 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:39.018519 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:39.018824 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:39.018827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:39.018829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:39.018997 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:39.019447 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:39.019453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:39.019728 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.019926 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.020264 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":260,"threads_started":1}
I20260811 06:21:39.022517 22597 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46625
I20260811 06:21:39.022753 22597 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:39.023404 22597 heartbeater.cc:511] Master 127.22.121.62:46625 requested a full tablet report, sending...
I20260811 06:21:39.024641 21707 ts_manager.cc:194] Registered new tserver with Master: bfbf31ff82754bcba1c61061f37fc82f (127.22.121.3:45973)
I20260811 06:21:39.025427 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:58255
I20260811 06:21:39.026556 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:39.041868 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.030939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.22.121.62:46625)
0811 06:21:39.031073 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.031077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.031532 (+ 455us) server_negotiation.cc:407] Connection header received
0811 06:21:39.031657 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.031660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.031713 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.031784 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:39.033044 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.033715 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.034954 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.035383 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.035474 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.037545 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:39.037549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:39.037552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:39.037580 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:39.039815 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.040387 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.040390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.040392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.040521 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.040820 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.040823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.040825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.041088 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:39.041257 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.041513 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.041659 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:21:39.045293 21707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:39.051932 21707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:39.055770 22094 heartbeater.cc:503] Master 127.22.121.62:46625 was elected leader, sending a full tablet report...
I20260811 06:21:39.074251 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.063455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.22.121.1:46607)
0811 06:21:39.065379 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.065387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.065414 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:21:39.065501 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.065508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.065721 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.066730 (+ 1009us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.066969 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.068611 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.071542 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.072941 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.073076 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.073288 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.073862 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.074017 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1756,"thread_start_us":2000,"threads_started":1}
I20260811 06:21:39.074446 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.063330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:46607 (local address 127.0.0.1:55508)
0811 06:21:39.064186 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.064238 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.064405 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.065970 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.065978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.066016 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.066485 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.066502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.068781 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.068787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.071362 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.071377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.073140 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.073146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.073618 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.073668 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.073795 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.073969 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":7,"spinlock_wait_cycles":2304,"thread_start_us":465,"threads_started":1}
I20260811 06:21:39.075395 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.063870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.22.121.2:45845)
0811 06:21:39.064963 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.064972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.065000 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:21:39.065105 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.065113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.065651 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.065848 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.066957 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.068630 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.072460 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.073657 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.073777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.073958 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.074905 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.075059 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":816,"threads_started":1}
I20260811 06:21:39.076151 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.063426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:45845 (local address 127.0.0.1:38940)
0811 06:21:39.064740 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.064756 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.064871 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.066110 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.066116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.066148 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.066736 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.066749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.068794 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.068800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.072322 (+ 3522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.072334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.074516 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.074522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.074697 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.074729 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.075650 (+ 921us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.075778 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":1189,"thread_start_us":1602,"threads_started":1}
I20260811 06:21:39.078189 21969 tablet_service.cc:1467] Processing CreateTablet for tablet e74f3815091846f594237c13e838b998 (DEFAULT_TABLE table=test-table [id=512298bb816c4e6b8ad44b89aa747bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:39.079182 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.064662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:45973 (local address 127.0.0.1:40906)
0811 06:21:39.070671 (+ 6009us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.070689 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.070804 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.071834 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.071839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.071865 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.072232 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.072242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.073947 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.073952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.076071 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.076083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.077520 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.077527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.077688 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.078711 (+ 1023us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.078778 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.078894 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5880,"thread_start_us":5750,"threads_started":1}
I20260811 06:21:39.079504 21969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74f3815091846f594237c13e838b998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:39.080288 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.065221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.22.121.3:45973)
0811 06:21:39.071338 (+ 6117us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.071345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.071376 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:21:39.071449 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.071455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.071653 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.071837 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.072440 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.073712 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.076219 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.077281 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.079544 (+ 2263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.079751 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.079822 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.079968 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5939,"thread_start_us":5878,"threads_started":1}
I20260811 06:21:39.079941 22226 tablet_service.cc:1467] Processing CreateTablet for tablet e74f3815091846f594237c13e838b998 (DEFAULT_TABLE table=test-table [id=512298bb816c4e6b8ad44b89aa747bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:39.081089 22226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74f3815091846f594237c13e838b998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:39.083828 22478 tablet_service.cc:1467] Processing CreateTablet for tablet e74f3815091846f594237c13e838b998 (DEFAULT_TABLE table=test-table [id=512298bb816c4e6b8ad44b89aa747bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:39.085114 22478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74f3815091846f594237c13e838b998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:39.090670 22634 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Bootstrap starting.
I20260811 06:21:39.093746 22634 tablet_bootstrap.cc:654] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:39.094982 22634 log.cc:824] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:39.095917 22635 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Bootstrap starting.
I20260811 06:21:39.098614 22642 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Bootstrap starting.
I20260811 06:21:39.099184 22635 tablet_bootstrap.cc:654] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:39.099527 22634 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: No bootstrap required, opened a new log
I20260811 06:21:39.099797 22634 ts_tablet_manager.cc:1397] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:21:39.100394 22635 log.cc:824] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:39.101562 22642 tablet_bootstrap.cc:654] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:39.102691 22642 log.cc:824] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:39.103456 22635 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: No bootstrap required, opened a new log
I20260811 06:21:39.103716 22635 ts_tablet_manager.cc:1397] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260811 06:21:39.104976 22642 tablet_bootstrap.cc:492] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: No bootstrap required, opened a new log
I20260811 06:21:39.105199 22642 ts_tablet_manager.cc:1397] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:21:39.109652 22634 raft_consensus.cc:348] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.110162 22634 raft_consensus.cc:374] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:39.110301 22634 raft_consensus.cc:729] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69305f6a89b9430c846530554a44b605, State: Initialized, Role: FOLLOWER
I20260811 06:21:39.111024 22634 consensus_queue.cc:260] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.112269 22642 raft_consensus.cc:348] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.112578 22642 raft_consensus.cc:374] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:39.112722 22642 raft_consensus.cc:729] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbf31ff82754bcba1c61061f37fc82f, State: Initialized, Role: FOLLOWER
I20260811 06:21:39.113453 22634 ts_tablet_manager.cc:1428] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260811 06:21:39.113348 22642 consensus_queue.cc:260] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.113528 22635 raft_consensus.cc:348] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.113988 22635 raft_consensus.cc:374] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:39.114130 22635 raft_consensus.cc:729] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffdcd112512435db65e9ae018c3c46f, State: Initialized, Role: FOLLOWER
I20260811 06:21:39.114445 22597 heartbeater.cc:503] Master 127.22.121.62:46625 was elected leader, sending a full tablet report...
I20260811 06:21:39.114811 22635 consensus_queue.cc:260] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.115053 22642 ts_tablet_manager.cc:1428] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 06:21:39.116150 22335 heartbeater.cc:503] Master 127.22.121.62:46625 was elected leader, sending a full tablet report...
I20260811 06:21:39.116755 22635 ts_tablet_manager.cc:1428] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
W20260811 06:21:39.253835 22598 tablet.cc:2369] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:39.270318 22651 raft_consensus.cc:482] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:39.270614 22651 raft_consensus.cc:504] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.272318 22651 leader_election.cc:288] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69305f6a89b9430c846530554a44b605 (127.22.121.1:46607), bfbf31ff82754bcba1c61061f37fc82f (127.22.121.3:45973)
I20260811 06:21:39.278220 22650 raft_consensus.cc:482] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 06:21:39.278477 22336 tablet.cc:2369] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:39.278512 22650 raft_consensus.cc:504] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
W20260811 06:21:39.279522 22095 tablet.cc:2369] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:39.279803 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.272777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:44123 (local address 127.22.121.3:45973)
0811 06:21:39.272976 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.272982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.273543 (+ 561us) server_negotiation.cc:407] Connection header received
0811 06:21:39.273772 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.273778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.273851 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.273916 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.274524 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.275295 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.277223 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.278067 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.278223 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.278297 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.279429 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.279561 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:21:39.280095 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.272642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:46607 (local address 127.22.121.2:52769)
0811 06:21:39.273569 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.273583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.273697 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.273981 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.273986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.274009 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.274421 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.274431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.275466 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.275470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.277052 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.277067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.278932 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.278939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.279169 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.279215 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.279578 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.279702 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":57,"thread_start_us":527,"threads_started":1}
I20260811 06:21:39.280366 22650 leader_election.cc:288] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69305f6a89b9430c846530554a44b605 (127.22.121.1:46607), 6ffdcd112512435db65e9ae018c3c46f (127.22.121.2:45845)
I20260811 06:21:39.280450 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.272670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:52769 (local address 127.22.121.1:46607)
0811 06:21:39.272883 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.272890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.273665 (+ 775us) server_negotiation.cc:407] Connection header received
0811 06:21:39.273810 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.273814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.273873 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.273956 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.274574 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.275324 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.277219 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.278077 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.278256 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.278326 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.279393 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.279527 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:21:39.280614 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.272595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:45973 (local address 127.22.121.2:44123)
0811 06:21:39.273434 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.273453 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.273604 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.273953 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.273957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.273982 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.274382 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.274392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.275411 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.275415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.277074 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.277088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.278902 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.278914 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.279214 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.279247 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.279969 (+ 722us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.280101 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":132,"thread_start_us":512,"threads_started":1}
I20260811 06:21:39.281340 22012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "6ffdcd112512435db65e9ae018c3c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69305f6a89b9430c846530554a44b605" is_pre_election: true
I20260811 06:21:39.281911 22012 raft_consensus.cc:2454] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ffdcd112512435db65e9ae018c3c46f in term 0.
I20260811 06:21:39.282132 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "6ffdcd112512435db65e9ae018c3c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbf31ff82754bcba1c61061f37fc82f" is_pre_election: true
I20260811 06:21:39.282599 22513 raft_consensus.cc:2454] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ffdcd112512435db65e9ae018c3c46f in term 0.
I20260811 06:21:39.282708 22132 leader_election.cc:302] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69305f6a89b9430c846530554a44b605, 6ffdcd112512435db65e9ae018c3c46f; no voters:
I20260811 06:21:39.283080 22651 raft_consensus.cc:2781] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:39.283210 22651 raft_consensus.cc:482] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:39.283327 22651 raft_consensus.cc:3037] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:39.286227 22651 raft_consensus.cc:504] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.287164 22012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "6ffdcd112512435db65e9ae018c3c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69305f6a89b9430c846530554a44b605"
I20260811 06:21:39.287168 22513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "6ffdcd112512435db65e9ae018c3c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbf31ff82754bcba1c61061f37fc82f"
I20260811 06:21:39.287370 22012 raft_consensus.cc:3037] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:39.287379 22513 raft_consensus.cc:3037] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:39.287803 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.280965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:46607 (local address 127.22.121.3:42351)
0811 06:21:39.281159 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.281177 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.281277 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.281854 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.281858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.281908 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.282099 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.282110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.283439 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.283445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.285391 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.285409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.287051 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.287061 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.287251 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.287308 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.287398 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.287528 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:21:39.288527 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.280959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:42351 (local address 127.22.121.1:46607)
0811 06:21:39.281479 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.281485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.281501 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:21:39.281554 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.281559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.281624 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.281699 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.282275 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.283244 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.285570 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.286751 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.286902 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.286967 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.288111 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.288241 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408}
I20260811 06:21:39.289003 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.281190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:43511 (local address 127.22.121.2:45845)
0811 06:21:39.281944 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.281949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.282667 (+ 718us) server_negotiation.cc:407] Connection header received
0811 06:21:39.282922 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.282927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.283003 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.283059 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:39.283993 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.284717 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.286783 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.287938 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.288056 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.288119 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.288666 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.288784 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626}
I20260811 06:21:39.289114 22009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "bfbf31ff82754bcba1c61061f37fc82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69305f6a89b9430c846530554a44b605" is_pre_election: true
I20260811 06:21:39.289700 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.280974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:45845 (local address 127.22.121.3:43511)
0811 06:21:39.282567 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:39.282588 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:39.282769 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:39.283238 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:39.283242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:39.283291 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:39.283834 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.283845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.284863 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.284867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.286603 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:39.286619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.288294 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.288302 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:39.288477 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.288510 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:39.289197 (+ 687us) client_negotiation.cc:240] Negotiation successful
0811 06:21:39.289330 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1469,"mutex_wait_us":382,"thread_start_us":2276,"threads_started":1}
I20260811 06:21:39.290488 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74f3815091846f594237c13e838b998" candidate_uuid: "bfbf31ff82754bcba1c61061f37fc82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ffdcd112512435db65e9ae018c3c46f" is_pre_election: true
I20260811 06:21:39.290911 22252 raft_consensus.cc:2379] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfbf31ff82754bcba1c61061f37fc82f in current term 1: Already voted for candidate 6ffdcd112512435db65e9ae018c3c46f in this term.
I20260811 06:21:39.291148 22012 raft_consensus.cc:2454] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ffdcd112512435db65e9ae018c3c46f in term 1.
I20260811 06:21:39.291421 22651 leader_election.cc:288] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [CANDIDATE]: Term 1 election: Requested vote from peers 69305f6a89b9430c846530554a44b605 (127.22.121.1:46607), bfbf31ff82754bcba1c61061f37fc82f (127.22.121.3:45973)
I20260811 06:21:39.291574 22132 leader_election.cc:302] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69305f6a89b9430c846530554a44b605, 6ffdcd112512435db65e9ae018c3c46f; no voters:
I20260811 06:21:39.291633 22513 raft_consensus.cc:2454] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ffdcd112512435db65e9ae018c3c46f in term 1.
I20260811 06:21:39.291697 22381 leader_election.cc:302] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfbf31ff82754bcba1c61061f37fc82f; no voters: 69305f6a89b9430c846530554a44b605, 6ffdcd112512435db65e9ae018c3c46f
I20260811 06:21:39.291836 22651 raft_consensus.cc:2781] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:39.292054 22650 raft_consensus.cc:2726] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:21:39.292346 22651 raft_consensus.cc:686] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 LEADER]: Becoming Leader. State: Replica: 6ffdcd112512435db65e9ae018c3c46f, State: Running, Role: LEADER
I20260811 06:21:39.292779 22651 consensus_queue.cc:237] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } }
I20260811 06:21:39.297091 21707 catalog_manager.cc:5495] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ffdcd112512435db65e9ae018c3c46f (127.22.121.2). New cstate: current_term: 1 leader_uuid: "6ffdcd112512435db65e9ae018c3c46f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ffdcd112512435db65e9ae018c3c46f" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 } health_report { overall_health: UNKNOWN } } }
I20260811 06:21:39.314540 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.309693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.22.121.2:45845)
0811 06:21:39.309870 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.309877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.309950 (+ 73us) server_negotiation.cc:407] Connection header received
0811 06:21:39.310101 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.310104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.310180 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.310300 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:39.310582 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.311291 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.312714 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.313107 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.313191 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.313604 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:39.314054 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:39.314156 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.314213 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.314271 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:21:39.321825 22012 raft_consensus.cc:1261] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 1 FOLLOWER]: Refusing update from remote peer 6ffdcd112512435db65e9ae018c3c46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:39.321880 22513 raft_consensus.cc:1261] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 1 FOLLOWER]: Refusing update from remote peer 6ffdcd112512435db65e9ae018c3c46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:39.322710 22657 consensus_queue.cc:1035] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "69305f6a89b9430c846530554a44b605" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:39.323032 22651 consensus_queue.cc:1035] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbf31ff82754bcba1c61061f37fc82f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 45973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:39.332806 22664 mvcc.cc:205] Tried to move back new op lower bound from 7317214410013528064 to 7317214409906503680. Current Snapshot: MvccSnapshot[applied={T|T < 7317214410013528064}]
I20260811 06:21:39.345993 21705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:39.346263 21705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:39.349756 21705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11ed029ed91c46cc851e7749e3a23d86: Sending DeleteTablet for 3 replicas of tablet e74f3815091846f594237c13e838b998
I20260811 06:21:39.350838 21969 tablet_service.cc:1514] Processing DeleteTablet for tablet e74f3815091846f594237c13e838b998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:39 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:55508
I20260811 06:21:39.350934 22226 tablet_service.cc:1514] Processing DeleteTablet for tablet e74f3815091846f594237c13e838b998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:39 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38940
I20260811 06:21:39.351538 22478 tablet_service.cc:1514] Processing DeleteTablet for tablet e74f3815091846f594237c13e838b998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:39 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:40906
I20260811 06:21:39.353544 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 21820
I20260811 06:21:39.355844 22680 tablet_replica.cc:330] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: stopping tablet replica
I20260811 06:21:39.356364 22681 tablet_replica.cc:330] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: stopping tablet replica
I20260811 06:21:39.356312 22680 raft_consensus.cc:2229] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:39.356706 22681 raft_consensus.cc:2229] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:39.356890 22680 raft_consensus.cc:2258] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:39.357487 22681 raft_consensus.cc:2258] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:39.357595 22678 tablet_replica.cc:330] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: stopping tablet replica
I20260811 06:21:39.357967 22678 raft_consensus.cc:2229] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:39.358466 22678 raft_consensus.cc:2258] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:39.359040 22680 ts_tablet_manager.cc:1905] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:39.360761 22678 ts_tablet_manager.cc:1905] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:39.362644 22681 ts_tablet_manager.cc:1905] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:39.366315 22680 ts_tablet_manager.cc:1918] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:39.366532 22680 log.cc:1196] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/e74f3815091846f594237c13e838b998
I20260811 06:21:39.367108 22680 ts_tablet_manager.cc:1939] T e74f3815091846f594237c13e838b998 P 6ffdcd112512435db65e9ae018c3c46f: Deleting consensus metadata
I20260811 06:21:39.367672 22678 ts_tablet_manager.cc:1918] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:39.367887 22678 log.cc:1196] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/e74f3815091846f594237c13e838b998
Aug 11 06:21:39 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:21:39.368517 22678 ts_tablet_manager.cc:1939] T e74f3815091846f594237c13e838b998 P 69305f6a89b9430c846530554a44b605: Deleting consensus metadata
I20260811 06:21:39.368889 21673 catalog_manager.cc:4841] TS 6ffdcd112512435db65e9ae018c3c46f (127.22.121.2:45845): tablet e74f3815091846f594237c13e838b998 (table test-table [id=512298bb816c4e6b8ad44b89aa747bb2]) successfully deleted
I20260811 06:21:39.370445 22681 ts_tablet_manager.cc:1918] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:39.370467 21671 catalog_manager.cc:4841] TS 69305f6a89b9430c846530554a44b605 (127.22.121.1:46607): tablet e74f3815091846f594237c13e838b998 (table test-table [id=512298bb816c4e6b8ad44b89aa747bb2]) successfully deleted
I20260811 06:21:39.370625 22681 log.cc:1196] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/e74f3815091846f594237c13e838b998
I20260811 06:21:39.371129 22681 ts_tablet_manager.cc:1939] T e74f3815091846f594237c13e838b998 P bfbf31ff82754bcba1c61061f37fc82f: Deleting consensus metadata
I20260811 06:21:39.372759 21672 catalog_manager.cc:4841] TS bfbf31ff82754bcba1c61061f37fc82f (127.22.121.3:45973): tablet e74f3815091846f594237c13e838b998 (table test-table [id=512298bb816c4e6b8ad44b89aa747bb2]) successfully deleted
I20260811 06:21:39.375764 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.356815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.22.121.1:46607)
0811 06:21:39.356994 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.357000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.357789 (+ 789us) server_negotiation.cc:407] Connection header received
0811 06:21:39.358079 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.358084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.358162 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.358241 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:39.360342 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.361439 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.363335 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.363957 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.364050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.369404 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:39.369418 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:39.369433 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:39.369476 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:39.373162 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.373828 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.373835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.373838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.373962 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.374320 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.374324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.374326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.374652 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:39.374945 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.375231 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.375471 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:39.376295 22087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:39.524637 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 21820
I20260811 06:21:39.551047 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 22101
Aug 11 06:21:39 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:21:39.575162 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.551796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.22.121.2:45845)
0811 06:21:39.551975 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.551981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.552110 (+ 129us) server_negotiation.cc:407] Connection header received
0811 06:21:39.552310 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.552318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.552380 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.552469 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:39.553822 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.554808 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.556390 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.556867 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.556978 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.569365 (+ 12387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:39.569376 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:39.569390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:39.569430 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:39.572362 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.573169 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.573175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.573177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.573313 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.573685 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.573689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.573691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.574082 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:39.574304 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.574607 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.574842 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:21:39.575889 22325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:39.685362 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 22101
I20260811 06:21:39.712491 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 22349
Aug 11 06:21:39 dist-test-slave-gjrz krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:21:39.726903 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:39.713211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.22.121.3:45973)
0811 06:21:39.713385 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:39.713391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:39.713413 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:21:39.713612 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:39.713617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:39.713680 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:39.713763 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:39.715073 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.715850 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.717134 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:39.717512 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:39.717632 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:39.721730 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:39.721744 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:39.721761 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:39.721795 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:39.724240 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.724872 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.724881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.724883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.725016 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:39.725362 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:39.725366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:39.725368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:39.725759 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:39.726054 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:39.726370 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 06:21:39.726586 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:21:39.727478 22591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:39.814899 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 22349
I20260811 06:21:39.832747 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 21633
I20260811 06:21:39.833796 21760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:39.940969 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 21633
2026-08-11T06:21:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4410 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T06:21:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:40Z Disabled control of system clock
I20260811 06:21:40.024456 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39083
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:40.341058 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:40.341305 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:40.341426 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:40.348357 22695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:40.348444 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:40.348502 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:40.348580 22695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:40.348639 22695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:40.357333 22695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:40.358429 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:40.360131 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:40.370369 22703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:40.371598 22704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:40.371685 22707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:40.372380 22695 server_base.cc:1034] running on GCE node
I20260811 06:21:40.373046 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:40.374290 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:40.375478 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786429300375434 us; error 37 us; skew 500 ppm
I20260811 06:21:40.375789 22695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:40.377835 22695 webserver.cc:460] Webserver started at http://127.22.121.62:43577/ using document root <none> and password file <none>
I20260811 06:21:40.378463 22695 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:40.378552 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:40.378840 22695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:40.381567 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "c564f6a0dac24bedbd62b936c21f4e1b"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "7a141faaaaf307a51056f98097b5ca31"
server_key_iv: "5209a20e41930c27300cc8f88c3ac9af"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.382354 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "c564f6a0dac24bedbd62b936c21f4e1b"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "7a141faaaaf307a51056f98097b5ca31"
server_key_iv: "5209a20e41930c27300cc8f88c3ac9af"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.387102 22695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:21:40.390297 22729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:40.391547 22695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:21:40.391721 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "c564f6a0dac24bedbd62b936c21f4e1b"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "7a141faaaaf307a51056f98097b5ca31"
server_key_iv: "5209a20e41930c27300cc8f88c3ac9af"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.391947 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:40.404327 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:40.405016 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:40.405260 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:40.421675 22695 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:39083
I20260811 06:21:40.421705 22825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:39083 every 8 connection(s)
I20260811 06:21:40.422821 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:40.424288 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 22695
I20260811 06:21:40.424432 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:40.424878 23012 external_mini_cluster.cc:1428] Setting key 503e358080d92d8f3a7cd3aabd9fe01b
I20260811 06:21:40.426630 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:40.439128 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:40.426587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.22.121.62:39083)
0811 06:21:40.427631 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:40.427644 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:40.427689 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:21:40.429562 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:40.429588 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:40.430145 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:40.430547 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:40.431045 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.433023 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.434342 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.435341 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.435509 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:40.436593 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:40.436603 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:40.436619 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:40.437292 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:40.437390 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:40.437420 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:40.437526 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:40.437718 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:21:40.438029 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":335,"threads_started":1}
I20260811 06:21:40.440125 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: Bootstrap starting.
I20260811 06:21:40.442989 22826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:40.443977 22826 log.cc:824] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:40.446327 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: No bootstrap required, opened a new log
I20260811 06:21:40.452957 22826 raft_consensus.cc:348] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } }
I20260811 06:21:40.453256 22826 raft_consensus.cc:374] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:40.453339 22826 raft_consensus.cc:729] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564f6a0dac24bedbd62b936c21f4e1b, State: Initialized, Role: FOLLOWER
I20260811 06:21:40.453850 22826 consensus_queue.cc:260] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } }
I20260811 06:21:40.454056 22826 raft_consensus.cc:388] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:40.454173 22826 raft_consensus.cc:482] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:40.454322 22826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:40.456346 22826 raft_consensus.cc:504] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } }
I20260811 06:21:40.456794 22826 leader_election.cc:302] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c564f6a0dac24bedbd62b936c21f4e1b; no voters:
I20260811 06:21:40.457293 22826 leader_election.cc:288] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:40.457412 22847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:40.458086 22847 raft_consensus.cc:686] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [term 1 LEADER]: Becoming Leader. State: Replica: c564f6a0dac24bedbd62b936c21f4e1b, State: Running, Role: LEADER
I20260811 06:21:40.458546 22847 consensus_queue.cc:237] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } }
I20260811 06:21:40.459059 22826 sys_catalog.cc:564] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:40.461092 22848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } } }
I20260811 06:21:40.461410 22848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:40.461831 22853 sys_catalog.cc:455] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c564f6a0dac24bedbd62b936c21f4e1b. Latest consensus state: current_term: 1 leader_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c564f6a0dac24bedbd62b936c21f4e1b" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39083 } } }
I20260811 06:21:40.462128 22853 sys_catalog.cc:458] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:40.464581 22860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:40.468726 22860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:40.477073 22860 catalog_manager.cc:1347] Generated new cluster ID: 12305e39f3d9423c9c877676dce103f6
I20260811 06:21:40.477176 22860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:40.511806 22860 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:40.513533 22860 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:40.527140 22860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: Generated new TSK 0
I20260811 06:21:40.528053 22860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:21:40.544173 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--builtin_ntp_servers=127.22.121.20:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:40.865630 22880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:40.865880 22880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:40.866003 22880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:40.873003 22880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:40.873098 22880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:40.873234 22880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:40.881870 22880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:40.882975 22880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:40.884626 22880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:40.897718 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:40.897770 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:40.897751 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:40.898634 22880 server_base.cc:1034] running on GCE node
I20260811 06:21:40.899134 22880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:40.899999 22880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:40.901190 22880 hybrid_clock.cc:648] HybridClock initialized: now 1786429300901150 us; error 32 us; skew 500 ppm
I20260811 06:21:40.901469 22880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:40.903501 22880 webserver.cc:460] Webserver started at http://127.22.121.1:33975/ using document root <none> and password file <none>
I20260811 06:21:40.904314 22880 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:40.904430 22880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:40.904723 22880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:40.907435 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "b0b64ef481dc4f8c811da8c0301d09dc"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "5123361b29d1b445b6cd63ceee8a0409"
server_key_iv: "31843374627557f5f1a4843232db191c"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.908186 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b64ef481dc4f8c811da8c0301d09dc"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "5123361b29d1b445b6cd63ceee8a0409"
server_key_iv: "31843374627557f5f1a4843232db191c"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.912894 22880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:21:40.916050 22898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:40.917220 22880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:21:40.917361 22880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "b0b64ef481dc4f8c811da8c0301d09dc"
format_stamp: "Formatted at 2026-08-11 06:21:40 on dist-test-slave-gjrz"
server_key: "5123361b29d1b445b6cd63ceee8a0409"
server_key_iv: "31843374627557f5f1a4843232db191c"
server_key_version: "encryptionkey@0"
I20260811 06:21:40.917555 22880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:40.937331 22880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:40.938042 22880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:40.938301 22880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:40.939183 22880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:40.940531 22880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:40.940627 22880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:40.940727 22880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:40.940797 22880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:40.969976 22880 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:35663
I20260811 06:21:40.970065 23116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:35663 every 8 connection(s)
I20260811 06:21:40.971122 22880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:40.971458 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 22880
I20260811 06:21:40.971621 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:40.972203 23012 external_mini_cluster.cc:1428] Setting key 7b091c3103fb9e6f9ce749e4c4a02e23
I20260811 06:21:40.975381 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--builtin_ntp_servers=127.22.121.20:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:40.983212 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:40.973642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:55271 (local address 127.22.121.62:39083)
0811 06:21:40.974113 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:40.974119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:40.975258 (+ 1139us) server_negotiation.cc:407] Connection header received
0811 06:21:40.977006 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:40.977012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:40.977091 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:40.977184 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:40.978202 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.978960 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.980515 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.980927 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.981039 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:40.982395 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:40.982401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:40.982404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:40.982421 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:40.982460 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:40.982472 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:40.982570 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:40.982770 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 06:21:40.982877 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":398,"threads_started":1}
I20260811 06:21:40.984145 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:40.974199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39083 (local address 127.22.121.1:55271)
0811 06:21:40.975111 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:40.975157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:40.976748 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:40.977417 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:40.977425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:40.977508 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:40.978041 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:40.978057 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.979109 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.979113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:40.980375 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:40.980387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:40.981139 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:40.981146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:40.981366 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:40.982084 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:40.982087 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:40.982124 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:40.982199 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:40.982572 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:40.982583 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:40.982591 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:21:40.982745 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 06:21:40.983116 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":303,"threads_started":1}
I20260811 06:21:40.985502 23117 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39083
I20260811 06:21:40.985807 23117 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:40.986675 23117 heartbeater.cc:511] Master 127.22.121.62:39083 requested a full tablet report, sending...
I20260811 06:21:40.989034 22758 ts_manager.cc:194] Registered new tserver with Master: b0b64ef481dc4f8c811da8c0301d09dc (127.22.121.1:35663)
I20260811 06:21:40.990974 22758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.1:55271
W20260811 06:21:41.296747 23121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:41.296993 23121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:41.297073 23121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:41.304066 23121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:41.304152 23121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:41.304267 23121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:41.312963 23121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:41.314082 23121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:41.315753 23121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:41.327975 23129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:41.327925 23131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:41.328003 23135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:41.328850 23121 server_base.cc:1034] running on GCE node
I20260811 06:21:41.329345 23121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:41.330176 23121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:41.331355 23121 hybrid_clock.cc:648] HybridClock initialized: now 1786429301331309 us; error 31 us; skew 500 ppm
I20260811 06:21:41.331620 23121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:41.333571 23121 webserver.cc:460] Webserver started at http://127.22.121.2:38153/ using document root <none> and password file <none>
I20260811 06:21:41.334178 23121 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:41.334259 23121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:41.334502 23121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:41.337110 23121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "ceaa94ffc77143aa8bcc73ec15197e8c"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0674177f99c650d90fb4a6b3c2d835e3"
server_key_iv: "281e5b1c40b45e850dd6b225e074d4ac"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.337795 23121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "ceaa94ffc77143aa8bcc73ec15197e8c"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0674177f99c650d90fb4a6b3c2d835e3"
server_key_iv: "281e5b1c40b45e850dd6b225e074d4ac"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.342350 23121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:21:41.345496 23144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.346627 23121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:21:41.346746 23121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "ceaa94ffc77143aa8bcc73ec15197e8c"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0674177f99c650d90fb4a6b3c2d835e3"
server_key_iv: "281e5b1c40b45e850dd6b225e074d4ac"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.346907 23121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:41.377028 23121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:41.377696 23121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:41.377940 23121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:41.378747 23121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:41.380163 23121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:41.380249 23121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.380326 23121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:41.380378 23121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.409197 23121 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:43999
I20260811 06:21:41.409262 23345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:43999 every 8 connection(s)
I20260811 06:21:41.410331 23121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:41.414626 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23121
I20260811 06:21:41.414812 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:41.415422 23012 external_mini_cluster.cc:1428] Setting key 2c5e3d55b3ec7af3259e8c99e8f21fc9
I20260811 06:21:41.418135 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--builtin_ntp_servers=127.22.121.20:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:41.423156 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.412530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:53077 (local address 127.22.121.62:39083)
0811 06:21:41.412695 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.412700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.413963 (+ 1263us) server_negotiation.cc:407] Connection header received
0811 06:21:41.416199 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.416204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.416275 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.416360 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:41.417447 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.418159 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.420313 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.420722 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.420836 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.422310 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:41.422317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:41.422322 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:41.422344 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:41.422384 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:41.422401 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:41.422496 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.422722 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.422816 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:41.424154 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.412993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39083 (local address 127.22.121.2:53077)
0811 06:21:41.413812 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:41.413852 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:41.415911 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:41.416571 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:41.416578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:41.416666 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:41.417273 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.417299 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.418326 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.418331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.420134 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.420150 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.420928 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.420935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.421191 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.421974 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:41.421977 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:41.422022 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:41.422109 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:41.422503 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:41.422512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:41.422519 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:21:41.422691 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 06:21:41.423019 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":258,"threads_started":1}
I20260811 06:21:41.425494 23347 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39083
I20260811 06:21:41.425783 23347 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:41.426574 23347 heartbeater.cc:511] Master 127.22.121.62:39083 requested a full tablet report, sending...
I20260811 06:21:41.428061 22758 ts_manager.cc:194] Registered new tserver with Master: ceaa94ffc77143aa8bcc73ec15197e8c (127.22.121.2:43999)
I20260811 06:21:41.429005 22758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.2:53077
W20260811 06:21:41.736989 23355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:41.737241 23355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:41.737357 23355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:41.744256 23355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:41.744346 23355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:41.744486 23355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:41.752959 23355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:41.754066 23355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:41.755707 23355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:41.767566 23361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:41.767748 23362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:41.767755 23365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:41.768579 23355 server_base.cc:1034] running on GCE node
I20260811 06:21:41.769127 23355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:41.769933 23355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:41.771135 23355 hybrid_clock.cc:648] HybridClock initialized: now 1786429301771089 us; error 33 us; skew 500 ppm
I20260811 06:21:41.771445 23355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:41.773481 23355 webserver.cc:460] Webserver started at http://127.22.121.3:40111/ using document root <none> and password file <none>
I20260811 06:21:41.774083 23355 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:41.774183 23355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:41.774466 23355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:41.777100 23355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "1d1fd665eb274c829f4be21581e7b71f"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0fde79f35c5758dfa7afd954348b1d98"
server_key_iv: "477e80e0791f342ae8756f576e474549"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.777823 23355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "1d1fd665eb274c829f4be21581e7b71f"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0fde79f35c5758dfa7afd954348b1d98"
server_key_iv: "477e80e0791f342ae8756f576e474549"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.782473 23355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:21:41.785567 23378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.786710 23355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:21:41.786850 23355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "1d1fd665eb274c829f4be21581e7b71f"
format_stamp: "Formatted at 2026-08-11 06:21:41 on dist-test-slave-gjrz"
server_key: "0fde79f35c5758dfa7afd954348b1d98"
server_key_iv: "477e80e0791f342ae8756f576e474549"
server_key_version: "encryptionkey@0"
I20260811 06:21:41.787035 23355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:41.807391 23355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:41.808077 23355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:41.808339 23355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:41.809166 23355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:41.810451 23355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:41.810546 23355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.810647 23355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:41.810722 23355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:41.839339 23355 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:36517
I20260811 06:21:41.839427 23566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:36517 every 8 connection(s)
I20260811 06:21:41.840463 23355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:41.844717 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23355
I20260811 06:21:41.844892 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:41.845356 23012 external_mini_cluster.cc:1428] Setting key 25f453d9767d72f58d85f37e1ea137b2
I20260811 06:21:41.851882 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.842691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:43391 (local address 127.22.121.62:39083)
0811 06:21:41.842859 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.842863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.844119 (+ 1256us) server_negotiation.cc:407] Connection header received
0811 06:21:41.845915 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.845922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.846013 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.846100 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:41.847080 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.847887 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.849393 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.849812 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.849919 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.851120 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:41.851125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:41.851128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:41.851143 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:41.851173 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:41.851184 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:41.851255 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.851486 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.851579 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:21:41.852687 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.843113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39083 (local address 127.22.121.3:43391)
0811 06:21:41.843946 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:41.844016 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:41.845661 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:41.846319 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:41.846327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:41.846405 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:41.846914 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.846930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.848039 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.848043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.849255 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.849268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.850027 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.850035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.850247 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.850826 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:41.850828 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:41.850864 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:41.850936 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:41.851287 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:41.851298 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:41.851306 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:21:41.851448 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 06:21:41.851785 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":275,"threads_started":1}
I20260811 06:21:41.853755 23567 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39083
I20260811 06:21:41.854007 23567 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:41.854660 23567 heartbeater.cc:511] Master 127.22.121.62:39083 requested a full tablet report, sending...
I20260811 06:21:41.855897 22758 ts_manager.cc:194] Registered new tserver with Master: 1d1fd665eb274c829f4be21581e7b71f (127.22.121.3:36517)
I20260811 06:21:41.856650 22758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.3:43391
I20260811 06:21:41.859757 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:41.868985 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.863519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.22.121.62:39083)
0811 06:21:41.863662 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.863667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.864132 (+ 465us) server_negotiation.cc:407] Connection header received
0811 06:21:41.864282 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.864287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.864357 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.864434 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:41.864891 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.865578 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.866944 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.867395 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.867499 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.868387 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:41.868392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:41.868395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:41.868410 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:41.868439 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:41.868450 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:41.868523 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.868652 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.868731 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:21:41.872109 22758 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:41.878453 22758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:41.912705 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.902007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.22.121.3:36517)
0811 06:21:41.904699 (+ 2692us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.904707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.905266 (+ 559us) server_negotiation.cc:407] Connection header received
0811 06:21:41.905594 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.905602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.905804 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.905951 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:41.906848 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.907872 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.910247 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.911548 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.911662 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.911921 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.912207 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.912365 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2488,"thread_start_us":2339,"threads_started":1}
I20260811 06:21:41.912724 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.904632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:36517 (local address 127.0.0.1:37974)
0811 06:21:41.905172 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:41.905187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:41.905408 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:41.906257 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:41.906261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:41.906283 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:41.906693 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.906702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.908024 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.908032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.910062 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.910075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.911718 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.911724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.912009 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.912048 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:21:41.912152 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:21:41.912324 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":252,"thread_start_us":281,"threads_started":1}
I20260811 06:21:41.913542 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.889105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.22.121.2:43999)
0811 06:21:41.903727 (+ 14622us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.903736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.904411 (+ 675us) server_negotiation.cc:407] Connection header received
0811 06:21:41.905283 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.905292 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.905522 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.905697 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:41.906624 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.908031 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.910684 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.912066 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.912192 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.912439 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.913055 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.913219 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14436,"thread_start_us":14633,"threads_started":1}
I20260811 06:21:41.913548 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.888919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:43999 (local address 127.0.0.1:58054)
0811 06:21:41.904295 (+ 15376us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:41.904313 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:41.904514 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:41.906000 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:41.906010 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:41.906035 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:41.906457 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.906469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.908156 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.908161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.910488 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.910499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.912592 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.912598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.912784 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.912899 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 06:21:41.912983 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 06:21:41.913194 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15253,"mutex_wait_us":210,"thread_start_us":11951,"threads_started":1}
I20260811 06:21:41.914373 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.888729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.22.121.1:35663)
0811 06:21:41.905046 (+ 16317us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:41.905053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:41.905080 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:21:41.905176 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:41.905183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:41.905433 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:41.905630 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:41.906315 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.907780 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.910684 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.912066 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.912192 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.912417 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:41.913920 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0811 06:21:41.914077 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16153,"thread_start_us":16817,"threads_started":1}
I20260811 06:21:41.914691 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:41.888570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45084)
0811 06:21:41.901283 (+ 12713us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:41.901336 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:41.901510 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:41.905800 (+ 4290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:41.905807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:41.905835 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:41.906137 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.906148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.907925 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.907931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.910436 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:41.910455 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:41.912574 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:41.912589 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:41.913122 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:41.913154 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:21:41.913230 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:21:41.913338 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12438,"mutex_wait_us":103,"thread_start_us":15456,"threads_started":1}
I20260811 06:21:41.916472 23455 tablet_service.cc:1467] Processing CreateTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd (DEFAULT_TABLE table=test-table [id=10ccd99e0c544fa19dfad3ac58bcadb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:41.916931 23231 tablet_service.cc:1467] Processing CreateTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd (DEFAULT_TABLE table=test-table [id=10ccd99e0c544fa19dfad3ac58bcadb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:41.917784 23455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9116b1abfe6f4d7987943fb383fb66dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:41.918292 23231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9116b1abfe6f4d7987943fb383fb66dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:41.918027 22983 tablet_service.cc:1467] Processing CreateTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd (DEFAULT_TABLE table=test-table [id=10ccd99e0c544fa19dfad3ac58bcadb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:41.919302 22983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9116b1abfe6f4d7987943fb383fb66dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:41.929309 23596 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Bootstrap starting.
I20260811 06:21:41.932253 23598 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Bootstrap starting.
I20260811 06:21:41.932701 23596 tablet_bootstrap.cc:654] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:41.933975 23596 log.cc:824] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:41.934787 23600 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Bootstrap starting.
I20260811 06:21:41.935392 23598 tablet_bootstrap.cc:654] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:41.936345 23598 log.cc:824] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:41.938063 23600 tablet_bootstrap.cc:654] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:41.938352 23596 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: No bootstrap required, opened a new log
I20260811 06:21:41.938642 23596 ts_tablet_manager.cc:1397] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260811 06:21:41.938994 23600 log.cc:824] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:41.941596 23598 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: No bootstrap required, opened a new log
I20260811 06:21:41.941776 23600 tablet_bootstrap.cc:492] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: No bootstrap required, opened a new log
I20260811 06:21:41.941826 23598 ts_tablet_manager.cc:1397] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 06:21:41.942036 23600 ts_tablet_manager.cc:1397] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:21:41.948501 23598 raft_consensus.cc:348] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.948462 23596 raft_consensus.cc:348] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.948822 23598 raft_consensus.cc:374] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:41.948884 23596 raft_consensus.cc:374] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:41.948923 23598 raft_consensus.cc:729] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaa94ffc77143aa8bcc73ec15197e8c, State: Initialized, Role: FOLLOWER
I20260811 06:21:41.948982 23596 raft_consensus.cc:729] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1fd665eb274c829f4be21581e7b71f, State: Initialized, Role: FOLLOWER
I20260811 06:21:41.949571 23598 consensus_queue.cc:260] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.949816 23596 consensus_queue.cc:260] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.951078 23347 heartbeater.cc:503] Master 127.22.121.62:39083 was elected leader, sending a full tablet report...
I20260811 06:21:41.951709 23598 ts_tablet_manager.cc:1428] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260811 06:21:41.952139 23600 raft_consensus.cc:348] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.952544 23600 raft_consensus.cc:374] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:41.952636 23600 raft_consensus.cc:729] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b64ef481dc4f8c811da8c0301d09dc, State: Initialized, Role: FOLLOWER
I20260811 06:21:41.953202 23567 heartbeater.cc:503] Master 127.22.121.62:39083 was elected leader, sending a full tablet report...
I20260811 06:21:41.953769 23596 ts_tablet_manager.cc:1428] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260811 06:21:41.954208 23600 consensus_queue.cc:260] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:41.955351 23117 heartbeater.cc:503] Master 127.22.121.62:39083 was elected leader, sending a full tablet report...
I20260811 06:21:41.956015 23600 ts_tablet_manager.cc:1428] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
W20260811 06:21:41.973237 23118 tablet.cc:2369] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:21:42.092025 23568 tablet.cc:2369] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:21:42.162211 23351 tablet.cc:2369] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:42.246587 23604 raft_consensus.cc:482] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:42.246991 23604 raft_consensus.cc:504] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:42.248764 23604 leader_election.cc:288] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b64ef481dc4f8c811da8c0301d09dc (127.22.121.1:35663), 1d1fd665eb274c829f4be21581e7b71f (127.22.121.3:36517)
I20260811 06:21:42.255610 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.249352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:44295 (local address 127.22.121.1:35663)
0811 06:21:42.249555 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.249561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.250292 (+ 731us) server_negotiation.cc:407] Connection header received
0811 06:21:42.250474 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.250481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.250557 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.250635 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:42.251236 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.251945 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.253863 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.254673 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.254823 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.254893 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.255200 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.255344 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:21:42.255666 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.249311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:57435 (local address 127.22.121.3:36517)
0811 06:21:42.249537 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.249544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.250196 (+ 652us) server_negotiation.cc:407] Connection header received
0811 06:21:42.250409 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.250413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.250473 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.250548 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:42.251138 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.251911 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.253955 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.254750 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.254892 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.254960 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.255313 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.255438 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:21:42.255749 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.249118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.22.121.2:44295)
0811 06:21:42.250199 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.250222 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:42.250344 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:42.250666 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:42.250670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:42.250709 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:42.251076 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:42.251088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.252079 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.252088 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:42.253672 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:42.253684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.254860 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.254867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:42.255043 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.255069 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:21:42.255156 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:21:42.255344 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":62,"thread_start_us":604,"threads_started":1}
I20260811 06:21:42.255793 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.249031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:36517 (local address 127.22.121.2:57435)
0811 06:21:42.250089 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.250107 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:42.250246 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:42.250575 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:42.250580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:42.250603 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:42.250988 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:42.250999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.252032 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.252038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:42.253677 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:42.253689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.254936 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.254942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:42.255106 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.255136 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:21:42.255212 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:21:42.255402 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":165,"thread_start_us":780,"threads_started":1}
I20260811 06:21:42.256726 23026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9116b1abfe6f4d7987943fb383fb66dd" candidate_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" is_pre_election: true
I20260811 06:21:42.257045 23489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9116b1abfe6f4d7987943fb383fb66dd" candidate_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1fd665eb274c829f4be21581e7b71f" is_pre_election: true
I20260811 06:21:42.257311 23026 raft_consensus.cc:2454] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa94ffc77143aa8bcc73ec15197e8c in term 0.
I20260811 06:21:42.257543 23489 raft_consensus.cc:2454] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaa94ffc77143aa8bcc73ec15197e8c in term 0.
I20260811 06:21:42.258044 23151 leader_election.cc:302] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b64ef481dc4f8c811da8c0301d09dc, ceaa94ffc77143aa8bcc73ec15197e8c; no voters:
I20260811 06:21:42.258380 23604 raft_consensus.cc:2781] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:42.258504 23604 raft_consensus.cc:482] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:42.258599 23604 raft_consensus.cc:3037] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:42.260826 23604 raft_consensus.cc:504] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:42.261382 23604 leader_election.cc:288] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [CANDIDATE]: Term 1 election: Requested vote from peers b0b64ef481dc4f8c811da8c0301d09dc (127.22.121.1:35663), 1d1fd665eb274c829f4be21581e7b71f (127.22.121.3:36517)
I20260811 06:21:42.261781 23026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9116b1abfe6f4d7987943fb383fb66dd" candidate_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b64ef481dc4f8c811da8c0301d09dc"
I20260811 06:21:42.261803 23489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9116b1abfe6f4d7987943fb383fb66dd" candidate_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1fd665eb274c829f4be21581e7b71f"
I20260811 06:21:42.261962 23026 raft_consensus.cc:3037] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:42.262024 23489 raft_consensus.cc:3037] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:42.264137 23026 raft_consensus.cc:2454] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa94ffc77143aa8bcc73ec15197e8c in term 1.
I20260811 06:21:42.264142 23489 raft_consensus.cc:2454] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaa94ffc77143aa8bcc73ec15197e8c in term 1.
I20260811 06:21:42.264487 23157 leader_election.cc:302] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1fd665eb274c829f4be21581e7b71f, ceaa94ffc77143aa8bcc73ec15197e8c; no voters:
I20260811 06:21:42.264782 23604 raft_consensus.cc:2781] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:42.265206 23604 raft_consensus.cc:686] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 1 LEADER]: Becoming Leader. State: Replica: ceaa94ffc77143aa8bcc73ec15197e8c, State: Running, Role: LEADER
I20260811 06:21:42.265650 23604 consensus_queue.cc:237] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } }
I20260811 06:21:42.270120 22758 catalog_manager.cc:5495] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaa94ffc77143aa8bcc73ec15197e8c (127.22.121.2). New cstate: current_term: 1 leader_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceaa94ffc77143aa8bcc73ec15197e8c" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 43999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 } health_report { overall_health: UNKNOWN } } }
I20260811 06:21:42.348309 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.343145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.22.121.2:43999)
0811 06:21:42.343343 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.343347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.343417 (+ 70us) server_negotiation.cc:407] Connection header received
0811 06:21:42.343611 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.343615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.343692 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.343816 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:42.344121 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.344882 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.346307 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.346710 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.346802 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.347300 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:42.347863 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:42.347980 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.348056 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.348117 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:21:42.356154 23489 raft_consensus.cc:1261] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 1 FOLLOWER]: Refusing update from remote peer ceaa94ffc77143aa8bcc73ec15197e8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:42.356153 23026 raft_consensus.cc:1261] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc [term 1 FOLLOWER]: Refusing update from remote peer ceaa94ffc77143aa8bcc73ec15197e8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:42.357132 23612 consensus_queue.cc:1035] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b64ef481dc4f8c811da8c0301d09dc" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:42.357440 23604 consensus_queue.cc:1035] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1fd665eb274c829f4be21581e7b71f" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 36517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:42.379984 22755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:42.380220 22755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:42.383577 22755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c564f6a0dac24bedbd62b936c21f4e1b: Sending DeleteTablet for 3 replicas of tablet 9116b1abfe6f4d7987943fb383fb66dd
I20260811 06:21:42.384624 22983 tablet_service.cc:1514] Processing DeleteTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:42 UTC) from {username='slave'} at 127.0.0.1:45084
I20260811 06:21:42.384783 23455 tablet_service.cc:1514] Processing DeleteTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:42 UTC) from {username='slave'} at 127.0.0.1:37974
I20260811 06:21:42.385346 23231 tablet_service.cc:1514] Processing DeleteTablet for tablet 9116b1abfe6f4d7987943fb383fb66dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:42 UTC) from {username='slave'} at 127.0.0.1:58054
I20260811 06:21:42.390868 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 22880
I20260811 06:21:42.401400 23646 tablet_replica.cc:330] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: stopping tablet replica
I20260811 06:21:42.401823 23646 raft_consensus.cc:2229] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:42.402503 23646 raft_consensus.cc:2258] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:42.404488 23646 ts_tablet_manager.cc:1905] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:42.412541 23645 tablet_replica.cc:330] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: stopping tablet replica
I20260811 06:21:42.413053 23645 raft_consensus.cc:2229] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:42.413638 23645 raft_consensus.cc:2258] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:42.417543 23645 ts_tablet_manager.cc:1905] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:42.419911 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.392697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.22.121.1:35663)
0811 06:21:42.392872 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.392878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.406995 (+ 14117us) server_negotiation.cc:407] Connection header received
0811 06:21:42.407467 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.407474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.407582 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.407656 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:42.408347 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.409674 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.413854 (+ 4180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.414535 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.414639 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.418110 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:42.418122 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:42.418140 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:42.418906 (+ 766us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:42.418981 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:42.419006 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:42.419092 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.419379 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.419500 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:21:42.421270 23105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:42.428709 23645 ts_tablet_manager.cc:1918] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:42.428954 23645 log.cc:1196] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/9116b1abfe6f4d7987943fb383fb66dd
I20260811 06:21:42.429520 23645 ts_tablet_manager.cc:1939] T 9116b1abfe6f4d7987943fb383fb66dd P 1d1fd665eb274c829f4be21581e7b71f: Deleting consensus metadata
I20260811 06:21:42.431464 22736 catalog_manager.cc:4841] TS 1d1fd665eb274c829f4be21581e7b71f (127.22.121.3:36517): tablet 9116b1abfe6f4d7987943fb383fb66dd (table test-table [id=10ccd99e0c544fa19dfad3ac58bcadb4]) successfully deleted
I20260811 06:21:42.438898 23646 ts_tablet_manager.cc:1918] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:42.439081 23646 log.cc:1196] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/9116b1abfe6f4d7987943fb383fb66dd
I20260811 06:21:42.439658 23646 ts_tablet_manager.cc:1939] T 9116b1abfe6f4d7987943fb383fb66dd P ceaa94ffc77143aa8bcc73ec15197e8c: Deleting consensus metadata
I20260811 06:21:42.441628 22736 catalog_manager.cc:4841] TS ceaa94ffc77143aa8bcc73ec15197e8c (127.22.121.2:43999): tablet 9116b1abfe6f4d7987943fb383fb66dd (table test-table [id=10ccd99e0c544fa19dfad3ac58bcadb4]) successfully deleted
I20260811 06:21:42.536649 23643 tablet_replica.cc:330] T 9116b1abfe6f4d7987943fb383fb66dd P b0b64ef481dc4f8c811da8c0301d09dc: stopping tablet replica
I20260811 06:21:42.536823 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 22880
W20260811 06:21:42.557804 22733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.121.1:35663 (error 108)
I20260811 06:21:42.558063 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23121
I20260811 06:21:42.560076 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.559154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45090)
0811 06:21:42.559644 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.559803 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":272,"threads_started":1}
W20260811 06:21:42.560535 22733 catalog_manager.cc:4584] TS b0b64ef481dc4f8c811da8c0301d09dc (127.22.121.1:35663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9116b1abfe6f4d7987943fb383fb66dd: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
I20260811 06:21:42.565567 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.558878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.22.121.2:43999)
0811 06:21:42.559065 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.559071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.559167 (+ 96us) server_negotiation.cc:407] Connection header received
0811 06:21:42.559461 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.559467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.559595 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.559701 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:42.560248 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.561163 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.562470 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.562905 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.563023 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.564147 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:42.564156 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:42.564173 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:42.564811 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:42.564875 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:42.564898 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:42.564999 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.565158 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.565252 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:42.566174 23338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:42.614081 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.613614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45100)
0811 06:21:42.613829 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.613926 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:21:42.615115 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.614715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45108)
0811 06:21:42.614860 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.614956 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 06:21:42.674731 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23121
I20260811 06:21:42.675582 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.675019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45116)
0811 06:21:42.675314 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.675409 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 06:21:42.676646 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.676293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45120)
0811 06:21:42.676454 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.676522 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:21:42.698133 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23355
I20260811 06:21:42.704777 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.698903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.22.121.3:36517)
0811 06:21:42.699083 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:42.699088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:42.699111 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:21:42.699330 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:42.699335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:42.699418 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:42.699513 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:42.699902 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.700625 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.701900 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:42.702306 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:42.702415 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:42.703500 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:42.703509 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:42.703524 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:42.704081 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:42.704134 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:42.704154 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:42.704250 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:42.704381 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 06:21:42.704492 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:42.705368 23556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:42.757107 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.756586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45124)
0811 06:21:42.756841 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.756940 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:21:42.758174 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:42.757720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:35663 (local address 127.0.0.1:45134)
0811 06:21:42.757935 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:42.758018 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 06:21:42.790602 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23355
I20260811 06:21:42.808511 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 22695
I20260811 06:21:42.809533 22815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:42.914963 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 22695
2026-08-11T06:21:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2972 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:21:43 dist-test-slave-gjrz krb5kdc[23671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:21:43 dist-test-slave-gjrz krb5kdc[23671](info): set up 2 sockets
Aug 11 06:21:43 dist-test-slave-gjrz krb5kdc[23671](info): commencing operation
krb5kdc: starting...
W20260811 06:21:44.158553 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:21:44 dist-test-slave-gjrz krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:21:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:21:44.392510 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45517
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:45517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:44.717798 23692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:44.718055 23692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:44.718146 23692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:44.724961 23692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:44.725049 23692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:44.725100 23692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:44.725159 23692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:44.725200 23692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:44.734180 23692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:45517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:44.735201 23692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:44.736335 23692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:44.747062 23697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:44.747139 23704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:44.747116 23702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:44.748190 23692 server_base.cc:1034] running on GCE node
I20260811 06:21:44.748879 23692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:44.750167 23692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:44.751351 23692 hybrid_clock.cc:648] HybridClock initialized: now 1786429304751304 us; error 37 us; skew 500 ppm
I20260811 06:21:44.751628 23692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:44 dist-test-slave-gjrz krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:44.754372 23692 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:21:44.755764 23692 webserver.cc:460] Webserver started at http://127.22.121.62:40009/ using document root <none> and password file <none>
I20260811 06:21:44.756363 23692 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:44.756480 23692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:44.756745 23692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:44.759536 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "e08b3dba0cfc42e2a23bc5e72546ba19"
format_stamp: "Formatted at 2026-08-11 06:21:44 on dist-test-slave-gjrz"
server_key: "14d4a5f06abdb4de62888c6131556e3e"
server_key_iv: "bc8b6f7975212a8232aa57357569c246"
server_key_version: "encryptionkey@0"
I20260811 06:21:44.760229 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "e08b3dba0cfc42e2a23bc5e72546ba19"
format_stamp: "Formatted at 2026-08-11 06:21:44 on dist-test-slave-gjrz"
server_key: "14d4a5f06abdb4de62888c6131556e3e"
server_key_iv: "bc8b6f7975212a8232aa57357569c246"
server_key_version: "encryptionkey@0"
I20260811 06:21:44.765300 23692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:21:44.768464 23713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:44.769613 23692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:44.769767 23692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "e08b3dba0cfc42e2a23bc5e72546ba19"
format_stamp: "Formatted at 2026-08-11 06:21:44 on dist-test-slave-gjrz"
server_key: "14d4a5f06abdb4de62888c6131556e3e"
server_key_iv: "bc8b6f7975212a8232aa57357569c246"
server_key_version: "encryptionkey@0"
I20260811 06:21:44.769932 23692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:44.785845 23692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:44.789323 23692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:44.789566 23692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:44.806243 23692 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:45517
I20260811 06:21:44.806275 23816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:45517 every 8 connection(s)
I20260811 06:21:44.807468 23692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:44.810948 23817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:44.811020 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23692
I20260811 06:21:44.811201 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:44.811800 23012 external_mini_cluster.cc:1428] Setting key 3efe8fda40979ef448a2a64b1b7f4414
I20260811 06:21:44.824610 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: Bootstrap starting.
Aug 11 06:21:44 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:44.827957 23817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:44.829258 23817 log.cc:824] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:44.832170 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: No bootstrap required, opened a new log
I20260811 06:21:44.834560 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:44.813533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.22.121.62:45517)
0811 06:21:44.814291 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:44.814301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:44.814345 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:21:44.815906 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:44.815926 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:44.816265 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:44.816586 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:44.818312 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:44.819575 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:44.821441 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:44.822164 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:44.822301 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:44.826815 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:44.826826 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:44.826843 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:44.826899 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:44.830396 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:44.831048 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:44.831057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:44.831064 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:44.831296 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:44.831633 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:44.831638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:44.831641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:44.832301 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:44.832570 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:44.832946 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 06:21:44.833332 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":352,"threads_started":1}
I20260811 06:21:44.840099 23817 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } }
I20260811 06:21:44.840402 23817 raft_consensus.cc:374] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:44.840477 23817 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08b3dba0cfc42e2a23bc5e72546ba19, State: Initialized, Role: FOLLOWER
I20260811 06:21:44.840914 23817 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } }
I20260811 06:21:44.841120 23817 raft_consensus.cc:388] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:44.841212 23817 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:44.841333 23817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:44.843349 23817 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } }
I20260811 06:21:44.843796 23817 leader_election.cc:302] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e08b3dba0cfc42e2a23bc5e72546ba19; no voters:
I20260811 06:21:44.844343 23817 leader_election.cc:288] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:44.844467 23826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:44.845072 23826 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [term 1 LEADER]: Becoming Leader. State: Replica: e08b3dba0cfc42e2a23bc5e72546ba19, State: Running, Role: LEADER
I20260811 06:21:44.845605 23826 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } }
I20260811 06:21:44.846307 23817 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:44.848243 23829 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } } }
I20260811 06:21:44.848747 23829 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:44.848553 23830 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08b3dba0cfc42e2a23bc5e72546ba19. Latest consensus state: current_term: 1 leader_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08b3dba0cfc42e2a23bc5e72546ba19" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45517 } } }
I20260811 06:21:44.849105 23830 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:44.856505 23841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:44.861275 23841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:44.869704 23841 catalog_manager.cc:1347] Generated new cluster ID: bda6d5cd689f4e8cad2f8832c5c5f8b8
I20260811 06:21:44.869807 23841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:44.883669 23841 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:44.885298 23841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:44.893932 23841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: Generated new TSK 0
I20260811 06:21:44.894837 23841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:21:44.966493 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:45.305718 23866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:45.305949 23866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:45.306077 23866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:45.312887 23866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:45.312980 23866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:45.313135 23866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:45.322157 23866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:45.323252 23866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:45.324527 23866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:45.337059 23874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:45.337086 23878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:45.337117 23875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:45.337844 23866 server_base.cc:1034] running on GCE node
I20260811 06:21:45.338385 23866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:45.339211 23866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:45.340430 23866 hybrid_clock.cc:648] HybridClock initialized: now 1786429305340388 us; error 36 us; skew 500 ppm
I20260811 06:21:45.340723 23866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:45 dist-test-slave-gjrz krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429305, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:45.343657 23866 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:21:45.344944 23866 webserver.cc:460] Webserver started at http://127.22.121.1:39701/ using document root <none> and password file <none>
I20260811 06:21:45.345526 23866 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:45.345623 23866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:45.345886 23866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:45.348690 23866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "e708aa7ca21142b6b35a720d8cbde094"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "906af0f0669b2e107956414a6bcfa779"
server_key_iv: "d71f10fd5fe555ab6b36e953aae09d07"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.349377 23866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "e708aa7ca21142b6b35a720d8cbde094"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "906af0f0669b2e107956414a6bcfa779"
server_key_iv: "d71f10fd5fe555ab6b36e953aae09d07"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.354305 23866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:21:45.357688 23890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.358870 23866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:21:45.359010 23866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "e708aa7ca21142b6b35a720d8cbde094"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "906af0f0669b2e107956414a6bcfa779"
server_key_iv: "d71f10fd5fe555ab6b36e953aae09d07"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.359174 23866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:45.376238 23866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:45.379719 23866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:45.379976 23866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:45.380825 23866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:45.382073 23866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:45.382153 23866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.382284 23866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:45.382339 23866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.410853 23866 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45637
I20260811 06:21:45.410907 24103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45637 every 8 connection(s)
I20260811 06:21:45.412039 23866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:45.412971 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 23866
I20260811 06:21:45.413151 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:45.413579 23012 external_mini_cluster.cc:1428] Setting key ba40dada4cb1043a537c6b6041e58d53
Aug 11 06:21:45 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429305, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:45.433010 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:45.414369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:32965 (local address 127.22.121.62:45517)
0811 06:21:45.414798 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:45.414802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:45.416031 (+ 1229us) server_negotiation.cc:407] Connection header received
0811 06:21:45.418337 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:45.418341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:45.418410 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:45.418495 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:45.420262 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.420968 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.422962 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.423375 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.423470 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:45.427130 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:45.427134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:45.427138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:45.427169 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:45.430142 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:45.430808 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:45.430813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:45.430817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:45.430961 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:45.431357 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:45.431362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:45.431366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:45.431641 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:45.431778 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:45.432454 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 06:21:45.432676 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":217,"threads_started":1}
I20260811 06:21:45.433820 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:45.414971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45517 (local address 127.22.121.1:32965)
0811 06:21:45.415817 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:45.415878 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:45.418081 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:45.418732 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:45.418742 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:45.419420 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:45.420116 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:45.420139 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.421120 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.421125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:45.422835 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:45.422851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.423590 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.423596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:45.423868 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:45.424479 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:45.424510 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:45.426926 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:45.430280 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:45.430286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:45.430297 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:45.430692 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:45.431062 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:45.431065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:45.431067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:45.431235 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:45.431769 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:45.431776 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:45.432080 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 06:21:45.432302 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 06:21:45.432759 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":358,"threads_started":1}
I20260811 06:21:45.435256 24104 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45517
I20260811 06:21:45.435578 24104 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:45.436398 24104 heartbeater.cc:511] Master 127.22.121.62:45517 requested a full tablet report, sending...
I20260811 06:21:45.438668 23745 ts_manager.cc:194] Registered new tserver with Master: e708aa7ca21142b6b35a720d8cbde094 (127.22.121.1:45637)
I20260811 06:21:45.440713 23745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:32965
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:21:45.474349 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:45.786067 24120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:45.786293 24120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:45.786417 24120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:45.793349 24120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:45.793440 24120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:45.793581 24120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:45.802613 24120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:45.803714 24120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:45.804872 24120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:45.817514 24126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:45.817535 24129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:45.818104 24133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:45.818989 24120 server_base.cc:1034] running on GCE node
I20260811 06:21:45.819583 24120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:45.820405 24120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:45.821604 24120 hybrid_clock.cc:648] HybridClock initialized: now 1786429305821551 us; error 45 us; skew 500 ppm
I20260811 06:21:45.821902 24120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:45 dist-test-slave-gjrz krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429305, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:45.824765 24120 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:21:45.826064 24120 webserver.cc:460] Webserver started at http://127.22.121.2:37643/ using document root <none> and password file <none>
I20260811 06:21:45.826633 24120 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:45.826730 24120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:45.826987 24120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:45.829612 24120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "09a9494eddc549a994971a044667f9eb"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "fd2d14c189bbb34b00f0a50b197ba85e"
server_key_iv: "8252957ccf6d8b309881d779ee11e6cd"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.830286 24120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "09a9494eddc549a994971a044667f9eb"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "fd2d14c189bbb34b00f0a50b197ba85e"
server_key_iv: "8252957ccf6d8b309881d779ee11e6cd"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.834959 24120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:21:45.838090 24141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.839243 24120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:45.839439 24120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "09a9494eddc549a994971a044667f9eb"
format_stamp: "Formatted at 2026-08-11 06:21:45 on dist-test-slave-gjrz"
server_key: "fd2d14c189bbb34b00f0a50b197ba85e"
server_key_iv: "8252957ccf6d8b309881d779ee11e6cd"
server_key_version: "encryptionkey@0"
I20260811 06:21:45.839601 24120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:45.877103 24120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:45.880635 24120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:45.880878 24120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:45.881745 24120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:45.882938 24120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:45.883018 24120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.883134 24120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:45.883184 24120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:45.913551 24120 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:35249
I20260811 06:21:45.913652 24361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:35249 every 8 connection(s)
I20260811 06:21:45.914690 24120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:45.924904 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24120
I20260811 06:21:45.925105 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:45.925683 23012 external_mini_cluster.cc:1428] Setting key d7073eeba39199612ada8f2133518274
Aug 11 06:21:45 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429305, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:45.933539 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:45.916925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:41327 (local address 127.22.121.62:45517)
0811 06:21:45.917155 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:45.917161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:45.918235 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 06:21:45.920000 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:45.920005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:45.920077 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:45.920167 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:45.921510 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.922380 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.923854 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.924305 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.924412 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:45.928269 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:45.928274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:45.928277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:45.928306 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:45.930692 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:45.931458 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:45.931461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:45.931464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:45.931593 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:45.932022 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:45.932026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:45.932029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:45.932318 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:45.932427 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:45.933146 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 06:21:45.933307 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 06:21:45.934650 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:45.917428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45517 (local address 127.22.121.2:41327)
0811 06:21:45.918088 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:45.918125 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:45.919749 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:45.920343 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:45.920349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:45.920821 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:45.921349 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:45.921366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.922507 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.922510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:45.923737 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:45.923750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:45.924482 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:45.924487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:45.924783 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:45.925442 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:45.925469 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:45.928060 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:45.930819 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:45.930828 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:45.930841 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:45.931357 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:45.931685 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:45.931689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:45.931692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:45.931902 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:45.932440 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:45.932447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:45.932787 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 06:21:45.933053 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 06:21:45.933484 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":246,"threads_started":1}
I20260811 06:21:45.935969 24363 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45517
I20260811 06:21:45.936269 24363 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:45.937009 24363 heartbeater.cc:511] Master 127.22.121.62:45517 requested a full tablet report, sending...
I20260811 06:21:45.938354 23745 ts_manager.cc:194] Registered new tserver with Master: 09a9494eddc549a994971a044667f9eb (127.22.121.2:35249)
I20260811 06:21:45.939121 23745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:41327
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:21:45.980214 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:46.292076 24368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:46.292342 24368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:46.292476 24368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:46.299540 24368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:46.299652 24368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:46.299803 24368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:46.309190 24368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:45517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:46.310333 24368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:46.311592 24368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:46.323840 24380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:46.323866 24381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:46.324014 24386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:46.324616 24368 server_base.cc:1034] running on GCE node
I20260811 06:21:46.325181 24368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:46.326033 24368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:46.327287 24368 hybrid_clock.cc:648] HybridClock initialized: now 1786429306327209 us; error 32 us; skew 500 ppm
I20260811 06:21:46.327598 24368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:46 dist-test-slave-gjrz krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:46.330585 24368 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:21:46.332034 24368 webserver.cc:460] Webserver started at http://127.22.121.3:34527/ using document root <none> and password file <none>
I20260811 06:21:46.332656 24368 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:46.332777 24368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:46.333140 24368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:46.335963 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "0537a2e16bf3402c965c53c98bd70fd2"
format_stamp: "Formatted at 2026-08-11 06:21:46 on dist-test-slave-gjrz"
server_key: "9465582a6028f5dc33e087af8344786b"
server_key_iv: "adf4c63e006782932847c0775d14d875"
server_key_version: "encryptionkey@0"
I20260811 06:21:46.336664 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "0537a2e16bf3402c965c53c98bd70fd2"
format_stamp: "Formatted at 2026-08-11 06:21:46 on dist-test-slave-gjrz"
server_key: "9465582a6028f5dc33e087af8344786b"
server_key_iv: "adf4c63e006782932847c0775d14d875"
server_key_version: "encryptionkey@0"
I20260811 06:21:46.341512 24368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:21:46.344969 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:46.346213 24368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:21:46.346352 24368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "0537a2e16bf3402c965c53c98bd70fd2"
format_stamp: "Formatted at 2026-08-11 06:21:46 on dist-test-slave-gjrz"
server_key: "9465582a6028f5dc33e087af8344786b"
server_key_iv: "adf4c63e006782932847c0775d14d875"
server_key_version: "encryptionkey@0"
I20260811 06:21:46.346527 24368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:46.360495 24368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:46.364514 24368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:46.364759 24368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:46.365594 24368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:46.366814 24368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:46.366886 24368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:46.366995 24368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:46.367050 24368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:46.396463 24368 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:35915
I20260811 06:21:46.396497 24604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:35915 every 8 connection(s)
I20260811 06:21:46.397607 24368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:46.400714 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24368
I20260811 06:21:46.400904 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:46.401479 23012 external_mini_cluster.cc:1428] Setting key be4f72004a02dff619caad85a96e5241
Aug 11 06:21:46 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:46.417484 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.399866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:54357 (local address 127.22.121.62:45517)
0811 06:21:46.400016 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.400020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.401282 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 06:21:46.403610 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.403616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.403698 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.403803 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:46.405268 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.406030 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.407513 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.407906 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.408001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.411038 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:46.411043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:46.411048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:46.411088 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:46.414343 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.415083 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.415089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.415092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.415291 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.415945 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.415950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.415953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.416335 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:46.416485 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.417094 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.417252 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:21:46.418201 24609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.400282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:45517 (local address 127.22.121.3:54357)
0811 06:21:46.401089 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.401147 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.403244 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.403970 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.403978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.404596 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:46.405119 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.405136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.406133 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.406137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.407407 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.407419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.408068 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.408073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.408249 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.408765 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:46.408785 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:46.410833 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:46.414472 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:46.414478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:46.414488 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:46.414962 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:46.415435 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:46.415440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:46.415443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:46.415676 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:46.416498 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:46.416506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:46.416781 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.416985 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.417306 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":305,"threads_started":1}
I20260811 06:21:46.419313 24606 heartbeater.cc:347] Connected to a master server at 127.22.121.62:45517
I20260811 06:21:46.419575 24606 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:46.420219 24606 heartbeater.cc:511] Master 127.22.121.62:45517 requested a full tablet report, sending...
I20260811 06:21:46.421463 23745 ts_manager.cc:194] Registered new tserver with Master: 0537a2e16bf3402c965c53c98bd70fd2 (127.22.121.3:35915)
I20260811 06:21:46.422231 23745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:54357
I20260811 06:21:46.428287 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:46.443559 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.432166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.22.121.62:45517)
0811 06:21:46.432296 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.432300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.432745 (+ 445us) server_negotiation.cc:407] Connection header received
0811 06:21:46.432856 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.432859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.432924 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.432998 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:46.434324 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.434984 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.436269 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.436652 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.436740 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.438945 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:46.438950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:46.438953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:46.438980 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:46.441268 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.441906 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.441909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.441911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.442024 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.442357 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.442360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.442362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.442644 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:46.442861 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.443162 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.443353 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:21:46.444092 24104 heartbeater.cc:503] Master 127.22.121.62:45517 was elected leader, sending a full tablet report...
I20260811 06:21:46.447139 23745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:46.453807 23745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:46.488174 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.465186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.22.121.1:45637)
0811 06:21:46.479242 (+ 14056us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.479249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.479296 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:21:46.479387 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.479394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.479645 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.479831 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:46.480911 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.482632 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.485659 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.487043 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.487171 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.487421 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.487705 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.487855 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13902,"thread_start_us":13747,"threads_started":1}
I20260811 06:21:46.488315 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.465049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52746)
0811 06:21:46.478091 (+ 13042us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.478131 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.478316 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.480105 (+ 1789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.480113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.480143 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:46.480663 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.480677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.482825 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.482830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.485523 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.485534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.487255 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.487282 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.487476 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.487537 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.487674 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.487842 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12784,"mutex_wait_us":101,"thread_start_us":14203,"threads_started":1}
I20260811 06:21:46.489022 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.465430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:35249 (local address 127.0.0.1:38808)
0811 06:21:46.479544 (+ 14114us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.479561 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.479682 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.480367 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.480372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.480401 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:46.480933 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.480947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.483083 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.483089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.485500 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.485521 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.488224 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.488231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.488429 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.488464 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.488552 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.488671 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13991,"mutex_wait_us":113,"thread_start_us":13476,"threads_started":1}
I20260811 06:21:46.489782 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.465511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.22.121.2:35249)
0811 06:21:46.478884 (+ 13373us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.478891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.479815 (+ 924us) server_negotiation.cc:407] Connection header received
0811 06:21:46.479916 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.479923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.480157 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.481151 (+ 994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:46.481398 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.482917 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.486174 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.487753 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.487872 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.488061 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.489332 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.489476 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13224,"thread_start_us":13103,"threads_started":1}
I20260811 06:21:46.492164 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.480478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35915 (local address 127.0.0.1:41816)
0811 06:21:46.485209 (+ 4731us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.485225 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.485341 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.485863 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.485868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.485894 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:46.486246 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.486259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.488075 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.488081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.489923 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.489938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.491416 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.491424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.491601 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.491634 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.491714 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.491832 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4596,"mutex_wait_us":1,"thread_start_us":4464,"threads_started":1}
I20260811 06:21:46.492596 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.479997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.22.121.3:35915)
0811 06:21:46.484915 (+ 4918us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.484923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.485323 (+ 400us) server_negotiation.cc:407] Connection header received
0811 06:21:46.485488 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.485496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.485695 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.485872 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:46.486462 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.487902 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.490093 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.491166 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.491858 (+ 692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.492076 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.492149 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.492288 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4746,"thread_start_us":4604,"threads_started":1}
I20260811 06:21:46.492443 23984 tablet_service.cc:1467] Processing CreateTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 (DEFAULT_TABLE table=test-table [id=aed3b9fabc464883816f2f34e38bd739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:46.493237 24226 tablet_service.cc:1467] Processing CreateTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 (DEFAULT_TABLE table=test-table [id=aed3b9fabc464883816f2f34e38bd739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:46.493925 23984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:46.494489 24226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:46.495995 24488 tablet_service.cc:1467] Processing CreateTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 (DEFAULT_TABLE table=test-table [id=aed3b9fabc464883816f2f34e38bd739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:46.497329 24488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:46.504789 24635 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Bootstrap starting.
I20260811 06:21:46.507701 24635 tablet_bootstrap.cc:654] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:46.508664 24636 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Bootstrap starting.
I20260811 06:21:46.509272 24635 log.cc:824] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:46.511961 24640 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Bootstrap starting.
I20260811 06:21:46.512351 24635 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: No bootstrap required, opened a new log
I20260811 06:21:46.512619 24635 ts_tablet_manager.cc:1397] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:21:46.512679 24636 tablet_bootstrap.cc:654] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:46.513863 24636 log.cc:824] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:46.514490 24640 tablet_bootstrap.cc:654] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:46.515600 24640 log.cc:824] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:46.516736 24636 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: No bootstrap required, opened a new log
I20260811 06:21:46.516976 24636 ts_tablet_manager.cc:1397] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 06:21:46.518749 24640 tablet_bootstrap.cc:492] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: No bootstrap required, opened a new log
I20260811 06:21:46.518967 24640 ts_tablet_manager.cc:1397] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 06:21:46.522260 24635 raft_consensus.cc:348] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.522666 24635 raft_consensus.cc:374] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:46.522801 24635 raft_consensus.cc:729] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e708aa7ca21142b6b35a720d8cbde094, State: Initialized, Role: FOLLOWER
I20260811 06:21:46.523422 24635 consensus_queue.cc:260] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.525043 24635 ts_tablet_manager.cc:1428] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260811 06:21:46.526266 24640 raft_consensus.cc:348] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.526670 24640 raft_consensus.cc:374] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:46.526767 24640 raft_consensus.cc:729] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0537a2e16bf3402c965c53c98bd70fd2, State: Initialized, Role: FOLLOWER
I20260811 06:21:46.527354 24640 consensus_queue.cc:260] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.527410 24636 raft_consensus.cc:348] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.527748 24636 raft_consensus.cc:374] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:46.527868 24636 raft_consensus.cc:729] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a9494eddc549a994971a044667f9eb, State: Initialized, Role: FOLLOWER
I20260811 06:21:46.528457 24636 consensus_queue.cc:260] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.529852 24606 heartbeater.cc:503] Master 127.22.121.62:45517 was elected leader, sending a full tablet report...
I20260811 06:21:46.530539 24640 ts_tablet_manager.cc:1428] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 06:21:46.531342 24363 heartbeater.cc:503] Master 127.22.121.62:45517 was elected leader, sending a full tablet report...
I20260811 06:21:46.531831 24636 ts_tablet_manager.cc:1428] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260811 06:21:46.541522 24646 raft_consensus.cc:482] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:46.541724 24646 raft_consensus.cc:504] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.543164 24646 leader_election.cc:288] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e708aa7ca21142b6b35a720d8cbde094 (127.22.121.1:45637), 0537a2e16bf3402c965c53c98bd70fd2 (127.22.121.3:35915)
I20260811 06:21:46.550653 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.543548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:49925 (local address 127.22.121.1:45637)
0811 06:21:46.543725 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.543732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.544414 (+ 682us) server_negotiation.cc:407] Connection header received
0811 06:21:46.544730 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.544735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.544809 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.544904 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:46.545332 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.546295 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.548705 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.549880 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.550057 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.550129 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.550367 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.550478 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:21:46.551008 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.543405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.22.121.2:49925)
0811 06:21:46.544304 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.544324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.544520 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.544916 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.544919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.544939 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:46.545216 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.545227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.546406 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.546410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.548564 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.548579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.550086 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.550091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.550207 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.550241 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.550342 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.550474 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":24,"thread_start_us":514,"threads_started":1}
I20260811 06:21:46.551179 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.543614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:55517 (local address 127.22.121.3:35915)
0811 06:21:46.543763 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.543768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.544603 (+ 835us) server_negotiation.cc:407] Connection header received
0811 06:21:46.545064 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.545068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.545117 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.545180 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:46.545711 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.546677 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.548967 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.550118 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.550242 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.550287 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.550896 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.550988 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:21:46.551247 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.543717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:35915 (local address 127.22.121.2:55517)
0811 06:21:46.544494 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.544519 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:46.544625 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:46.545229 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:46.545233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:46.545255 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:46.545570 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.545579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.546795 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.546801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.548834 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:46.548843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.550326 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.550332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:46.550534 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.550566 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:21:46.550725 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 06:21:46.550839 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":3,"thread_start_us":452,"threads_started":1}
I20260811 06:21:46.551963 24021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a3a71f5cb54c8fa5ce7fee7c8d9d13" candidate_uuid: "09a9494eddc549a994971a044667f9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708aa7ca21142b6b35a720d8cbde094" is_pre_election: true
I20260811 06:21:46.552121 24529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a3a71f5cb54c8fa5ce7fee7c8d9d13" candidate_uuid: "09a9494eddc549a994971a044667f9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0537a2e16bf3402c965c53c98bd70fd2" is_pre_election: true
I20260811 06:21:46.552369 24021 raft_consensus.cc:2454] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a9494eddc549a994971a044667f9eb in term 0.
I20260811 06:21:46.552556 24529 raft_consensus.cc:2454] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09a9494eddc549a994971a044667f9eb in term 0.
I20260811 06:21:46.552953 24142 leader_election.cc:302] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a9494eddc549a994971a044667f9eb, e708aa7ca21142b6b35a720d8cbde094; no voters:
I20260811 06:21:46.553324 24646 raft_consensus.cc:2781] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:46.553440 24646 raft_consensus.cc:482] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:46.553534 24646 raft_consensus.cc:3037] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:46.555625 24646 raft_consensus.cc:504] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.556149 24646 leader_election.cc:288] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [CANDIDATE]: Term 1 election: Requested vote from peers e708aa7ca21142b6b35a720d8cbde094 (127.22.121.1:45637), 0537a2e16bf3402c965c53c98bd70fd2 (127.22.121.3:35915)
I20260811 06:21:46.556521 24021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a3a71f5cb54c8fa5ce7fee7c8d9d13" candidate_uuid: "09a9494eddc549a994971a044667f9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e708aa7ca21142b6b35a720d8cbde094"
I20260811 06:21:46.556576 24529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a3a71f5cb54c8fa5ce7fee7c8d9d13" candidate_uuid: "09a9494eddc549a994971a044667f9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0537a2e16bf3402c965c53c98bd70fd2"
I20260811 06:21:46.556708 24021 raft_consensus.cc:3037] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:46.556746 24529 raft_consensus.cc:3037] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:46.558907 24021 raft_consensus.cc:2454] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a9494eddc549a994971a044667f9eb in term 1.
I20260811 06:21:46.558988 24529 raft_consensus.cc:2454] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09a9494eddc549a994971a044667f9eb in term 1.
I20260811 06:21:46.559302 24142 leader_election.cc:302] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09a9494eddc549a994971a044667f9eb, e708aa7ca21142b6b35a720d8cbde094; no voters:
I20260811 06:21:46.559619 24646 raft_consensus.cc:2781] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:46.559988 24646 raft_consensus.cc:686] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 1 LEADER]: Becoming Leader. State: Replica: 09a9494eddc549a994971a044667f9eb, State: Running, Role: LEADER
I20260811 06:21:46.560374 24646 consensus_queue.cc:237] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } }
I20260811 06:21:46.564412 23745 catalog_manager.cc:5495] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 09a9494eddc549a994971a044667f9eb (127.22.121.2). New cstate: current_term: 1 leader_uuid: "09a9494eddc549a994971a044667f9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09a9494eddc549a994971a044667f9eb" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 35249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 } health_report { overall_health: UNKNOWN } } }
I20260811 06:21:46.586668 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.581870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.22.121.2:35249)
0811 06:21:46.582079 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.582086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.582171 (+ 85us) server_negotiation.cc:407] Connection header received
0811 06:21:46.582311 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.582314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.582389 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.582495 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:46.582804 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.583545 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.584796 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.585192 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.585278 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.585694 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:46.586102 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:46.586219 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.586378 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.586413 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:21:46.593957 24021 raft_consensus.cc:1261] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 1 FOLLOWER]: Refusing update from remote peer 09a9494eddc549a994971a044667f9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:46.593961 24529 raft_consensus.cc:1261] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 1 FOLLOWER]: Refusing update from remote peer 09a9494eddc549a994971a044667f9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:46.594942 24654 consensus_queue.cc:1035] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e708aa7ca21142b6b35a720d8cbde094" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:46.595255 24646 consensus_queue.cc:1035] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0537a2e16bf3402c965c53c98bd70fd2" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 35915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:46.606287 24665 mvcc.cc:205] Tried to move back new op lower bound from 7317214439799324672 to 7317214439674503168. Current Snapshot: MvccSnapshot[applied={T|T < 7317214439799324672}]
I20260811 06:21:46.606586 24664 mvcc.cc:205] Tried to move back new op lower bound from 7317214439799324672 to 7317214439674503168. Current Snapshot: MvccSnapshot[applied={T|T < 7317214439799324672}]
I20260811 06:21:46.606760 24662 mvcc.cc:205] Tried to move back new op lower bound from 7317214439799324672 to 7317214439674503168. Current Snapshot: MvccSnapshot[applied={T|T < 7317214439799324672}]
I20260811 06:21:46.617827 23742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:46.618125 23742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:46.621726 23742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e08b3dba0cfc42e2a23bc5e72546ba19: Sending DeleteTablet for 3 replicas of tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13
I20260811 06:21:46.622694 23984 tablet_service.cc:1514] Processing DeleteTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:46 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:52746
I20260811 06:21:46.622862 24488 tablet_service.cc:1514] Processing DeleteTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:46 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:41816
I20260811 06:21:46.622941 24226 tablet_service.cc:1514] Processing DeleteTablet for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:46 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:38808
I20260811 06:21:46.628209 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23866
I20260811 06:21:46.635713 24684 tablet_replica.cc:330] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: stopping tablet replica
I20260811 06:21:46.636133 24684 raft_consensus.cc:2229] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:46.636761 24684 raft_consensus.cc:2258] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:46.638813 24684 ts_tablet_manager.cc:1905] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:46.645013 24682 tablet_replica.cc:330] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: stopping tablet replica
I20260811 06:21:46.645558 24682 raft_consensus.cc:2229] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:46.645866 24684 ts_tablet_manager.cc:1918] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:46.646113 24682 raft_consensus.cc:2258] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:46.646083 24684 log.cc:1196] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/11a3a71f5cb54c8fa5ce7fee7c8d9d13
I20260811 06:21:46.646665 24684 ts_tablet_manager.cc:1939] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 09a9494eddc549a994971a044667f9eb: Deleting consensus metadata
I20260811 06:21:46.646941 24680 tablet_replica.cc:330] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: stopping tablet replica
I20260811 06:21:46.647368 24680 raft_consensus.cc:2229] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:46.647817 24680 raft_consensus.cc:2258] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:46.648644 23719 catalog_manager.cc:4841] TS 09a9494eddc549a994971a044667f9eb (127.22.121.2:35249): tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 (table test-table [id=aed3b9fabc464883816f2f34e38bd739]) successfully deleted
Aug 11 06:21:46 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:21:46.649823 24682 ts_tablet_manager.cc:1905] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:46.652158 24680 ts_tablet_manager.cc:1905] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:46.656749 24682 ts_tablet_manager.cc:1918] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:46.656948 24682 log.cc:1196] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/11a3a71f5cb54c8fa5ce7fee7c8d9d13
I20260811 06:21:46.657505 24682 ts_tablet_manager.cc:1939] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P 0537a2e16bf3402c965c53c98bd70fd2: Deleting consensus metadata
I20260811 06:21:46.657882 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.629600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.22.121.1:45637)
0811 06:21:46.629789 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.629796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.635924 (+ 6128us) server_negotiation.cc:407] Connection header received
0811 06:21:46.636204 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.636210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.636289 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.636369 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:46.638181 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.639331 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.641175 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.641794 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.641900 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.650431 (+ 8531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:46.650442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:46.650458 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:46.650503 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:46.654383 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.655211 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.655219 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.655223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.655395 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.655871 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.655876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.655879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.656373 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:46.656659 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.657250 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.657497 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:21:46.659098 24680 ts_tablet_manager.cc:1918] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:46.659159 23719 catalog_manager.cc:4841] TS 0537a2e16bf3402c965c53c98bd70fd2 (127.22.121.3:35915): tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13 (table test-table [id=aed3b9fabc464883816f2f34e38bd739]) successfully deleted
I20260811 06:21:46.659473 24092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:46.659611 24680 log.cc:1196] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/11a3a71f5cb54c8fa5ce7fee7c8d9d13
I20260811 06:21:46.745702 24680 ts_tablet_manager.cc:1939] T 11a3a71f5cb54c8fa5ce7fee7c8d9d13 P e708aa7ca21142b6b35a720d8cbde094: Deleting consensus metadata
I20260811 06:21:46.747694 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23866
I20260811 06:21:46.776189 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24120
W20260811 06:21:46.776199 23714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.121.1:45637 (error 108)
I20260811 06:21:46.778124 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.777654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52764)
0811 06:21:46.777880 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.777977 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 06:21:46.778417 23714 catalog_manager.cc:4584] TS e708aa7ca21142b6b35a720d8cbde094 (127.22.121.1:45637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 11a3a71f5cb54c8fa5ce7fee7c8d9d13: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Aug 11 06:21:46 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:21:46.793829 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.777387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.22.121.2:35249)
0811 06:21:46.777644 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.777650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.777723 (+ 73us) server_negotiation.cc:407] Connection header received
0811 06:21:46.777926 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.777933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.778033 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.778162 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:46.779731 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.780965 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.782212 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.782896 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.783036 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.786981 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:46.786997 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:46.787021 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:46.787069 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:46.790821 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.791614 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.791624 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.791628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.791801 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.792220 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.792225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.792229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.792687 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:46.793035 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.793246 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.793462 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 06:21:46.794621 24349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:46.831766 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.831296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52772)
0811 06:21:46.831486 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.831592 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:21:46.832718 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.832359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52776)
0811 06:21:46.832492 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.832576 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 06:21:46.893239 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.892666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52784)
0811 06:21:46.892899 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.893007 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:21:46.894240 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.893868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52794)
0811 06:21:46.894008 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.894102 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 06:21:46.905818 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24120
I20260811 06:21:46.930298 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24368
Aug 11 06:21:46 dist-test-slave-gjrz krb5kdc[23671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:21:46.944937 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.931019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.22.121.3:35915)
0811 06:21:46.931205 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:46.931211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:46.931231 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:21:46.931452 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:46.931458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:46.931528 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:46.931624 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:46.932970 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.933714 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.934963 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:46.935518 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:46.935640 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:46.939717 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:46.939730 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:46.939746 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:46.939777 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:46.942253 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.942885 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.942893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.942895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.943033 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:46.943504 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:46.943509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:46.943512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:46.943912 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:46.944160 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:46.944460 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 06:21:46.944645 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:21:46.945582 24597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:46.975252 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.974674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52798)
0811 06:21:46.974919 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.975035 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:21:46.976338 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:46.975908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52804)
0811 06:21:46.976117 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:46.976201 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 06:21:47.033051 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24368
I20260811 06:21:47.055212 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 23692
I20260811 06:21:47.056367 23802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:47.162644 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:47.162010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45637 (local address 127.0.0.1:52806)
0811 06:21:47.162216 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:47.162343 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:45637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:21:47.162845 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 23692
2026-08-11T06:21:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4248 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T06:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:47Z Disabled control of system clock
I20260811 06:21:47.247085 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46227
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:47.575821 24701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:47.576090 24701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:47.576211 24701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:47.583150 24701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:47.583244 24701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:47.583329 24701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:47.583402 24701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:47.583460 24701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:47.592193 24701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:46227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:47.593314 24701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:47.594998 24701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:47.605985 24717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:47.606042 24707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:47.606228 24706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:47.606964 24701 server_base.cc:1034] running on GCE node
I20260811 06:21:47.607692 24701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:47.608990 24701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:47.610184 24701 hybrid_clock.cc:648] HybridClock initialized: now 1786429307610136 us; error 40 us; skew 500 ppm
I20260811 06:21:47.610533 24701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:47.612571 24701 webserver.cc:460] Webserver started at http://127.22.121.62:44345/ using document root <none> and password file <none>
I20260811 06:21:47.613169 24701 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:47.613256 24701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:47.613523 24701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:47.616240 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "c10a5b81ebc642f09c93d2a96a9c8bde"
format_stamp: "Formatted at 2026-08-11 06:21:47 on dist-test-slave-gjrz"
server_key: "d54797c1bb4b793bf9c1ed2644d7593e"
server_key_iv: "a09fd7e4c2491dc642d306fad9523113"
server_key_version: "encryptionkey@0"
I20260811 06:21:47.616969 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "c10a5b81ebc642f09c93d2a96a9c8bde"
format_stamp: "Formatted at 2026-08-11 06:21:47 on dist-test-slave-gjrz"
server_key: "d54797c1bb4b793bf9c1ed2644d7593e"
server_key_iv: "a09fd7e4c2491dc642d306fad9523113"
server_key_version: "encryptionkey@0"
I20260811 06:21:47.621745 24701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:21:47.624941 24724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:47.626204 24701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:21:47.626384 24701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "c10a5b81ebc642f09c93d2a96a9c8bde"
format_stamp: "Formatted at 2026-08-11 06:21:47 on dist-test-slave-gjrz"
server_key: "d54797c1bb4b793bf9c1ed2644d7593e"
server_key_iv: "a09fd7e4c2491dc642d306fad9523113"
server_key_version: "encryptionkey@0"
I20260811 06:21:47.626597 24701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:47.637943 24701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:47.638700 24701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:47.638949 24701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:47.655938 24701 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:46227
I20260811 06:21:47.655972 24827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:46227 every 8 connection(s)
I20260811 06:21:47.657110 24701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:47.658820 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24701
I20260811 06:21:47.658974 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:47.659507 23012 external_mini_cluster.cc:1428] Setting key ff6dbdeb91615311d3ebc70c6efd7314
I20260811 06:21:47.660956 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:47.673779 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:47.661224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.22.121.62:46227)
0811 06:21:47.662300 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:47.662314 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:47.662365 (+ 51us) server_negotiation.cc:407] Connection header received
0811 06:21:47.664158 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:47.664184 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:47.664755 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:47.665161 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:47.665673 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:47.667684 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:47.669014 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:47.669987 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:47.670163 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:47.671193 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:47.671203 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:47.671219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:47.671939 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:47.672027 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:47.672053 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:47.672181 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:47.672313 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:21:47.672628 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":350,"threads_started":1}
I20260811 06:21:47.675105 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: Bootstrap starting.
I20260811 06:21:47.678069 24828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:47.678982 24828 log.cc:824] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:47.681567 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: No bootstrap required, opened a new log
I20260811 06:21:47.688551 24828 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } }
I20260811 06:21:47.688838 24828 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:47.688910 24828 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10a5b81ebc642f09c93d2a96a9c8bde, State: Initialized, Role: FOLLOWER
I20260811 06:21:47.689388 24828 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } }
I20260811 06:21:47.689584 24828 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:47.689682 24828 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:47.689813 24828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:47.691785 24828 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } }
I20260811 06:21:47.692265 24828 leader_election.cc:302] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10a5b81ebc642f09c93d2a96a9c8bde; no voters:
I20260811 06:21:47.692874 24828 leader_election.cc:288] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:47.692978 24837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:47.693629 24837 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [term 1 LEADER]: Becoming Leader. State: Replica: c10a5b81ebc642f09c93d2a96a9c8bde, State: Running, Role: LEADER
I20260811 06:21:47.694142 24837 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } }
I20260811 06:21:47.694658 24828 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:47.696890 24838 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } } }
I20260811 06:21:47.697328 24838 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:47.696888 24839 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10a5b81ebc642f09c93d2a96a9c8bde. Latest consensus state: current_term: 1 leader_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a5b81ebc642f09c93d2a96a9c8bde" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 46227 } } }
I20260811 06:21:47.697849 24839 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:47.698884 24850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:47.703737 24850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:47.712170 24850 catalog_manager.cc:1347] Generated new cluster ID: f6472af7f1874e9f9fc3f7e742f7b372
I20260811 06:21:47.712275 24850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:47.738355 24850 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:47.739727 24850 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:47.744725 24850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: Generated new TSK 0
I20260811 06:21:47.745412 24850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:21:47.753412 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--builtin_ntp_servers=127.22.121.20:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:48.113556 24875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:48.113796 24875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:48.113900 24875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:48.120941 24875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:48.121028 24875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:48.121160 24875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:48.130014 24875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:48.131229 24875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:48.133003 24875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:48.147292 24882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:48.147254 24884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:48.148450 24891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:48.148507 24875 server_base.cc:1034] running on GCE node
I20260811 06:21:48.149158 24875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:48.150063 24875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:48.151315 24875 hybrid_clock.cc:648] HybridClock initialized: now 1786429308151216 us; error 60 us; skew 500 ppm
I20260811 06:21:48.151625 24875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:48.153867 24875 webserver.cc:460] Webserver started at http://127.22.121.1:33865/ using document root <none> and password file <none>
I20260811 06:21:48.154559 24875 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:48.154685 24875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:48.155025 24875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:48.157872 24875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "64737a3a2db540ea946db4bfcaf6f925"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "f50df4b5d29505f6e393d6bbc0e460de"
server_key_iv: "ccc107cd2c5c3f2ad405241dae25cb96"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.158639 24875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "64737a3a2db540ea946db4bfcaf6f925"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "f50df4b5d29505f6e393d6bbc0e460de"
server_key_iv: "ccc107cd2c5c3f2ad405241dae25cb96"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.163650 24875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 06:21:48.167071 24902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.168498 24875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:21:48.168654 24875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "64737a3a2db540ea946db4bfcaf6f925"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "f50df4b5d29505f6e393d6bbc0e460de"
server_key_iv: "ccc107cd2c5c3f2ad405241dae25cb96"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.168855 24875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:48.188421 24875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:48.189185 24875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:48.189455 24875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:48.190394 24875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:48.191877 24875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:48.191980 24875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.192078 24875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:48.192145 24875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.231462 24875 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:33511
I20260811 06:21:48.231585 25088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:33511 every 8 connection(s)
I20260811 06:21:48.232719 24875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:48.236392 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 24875
I20260811 06:21:48.236588 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:48.237131 23012 external_mini_cluster.cc:1428] Setting key df27de9ff8bf2fdcc9b9fc91eace4af4
I20260811 06:21:48.240291 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--builtin_ntp_servers=127.22.121.20:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:48.247809 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:48.235299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:33159 (local address 127.22.121.62:46227)
0811 06:21:48.235741 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:48.235745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:48.237518 (+ 1773us) server_negotiation.cc:407] Connection header received
0811 06:21:48.239557 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:48.239562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:48.239650 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:48.239763 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:48.241233 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.242271 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.244599 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.245193 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.245316 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:48.246923 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:48.246929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:48.246933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:48.246956 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:48.247003 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:48.247018 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:48.247105 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:48.247349 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 06:21:48.247446 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":216,"threads_started":1}
I20260811 06:21:48.249009 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:48.236045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46227 (local address 127.22.121.1:33159)
0811 06:21:48.237267 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:48.237320 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:48.239158 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:48.240050 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:48.240063 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:48.240188 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:48.241028 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:48.241055 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.242439 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.242446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:48.244443 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:48.244460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.245437 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.245446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:48.245773 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:48.246554 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:48.246558 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:48.246611 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:48.246719 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:48.247116 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:48.247126 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:48.247134 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:21:48.247337 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 06:21:48.247744 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":351,"threads_started":1}
I20260811 06:21:48.250517 25092 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46227
I20260811 06:21:48.250847 25092 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:48.251765 25092 heartbeater.cc:511] Master 127.22.121.62:46227 requested a full tablet report, sending...
I20260811 06:21:48.253783 24755 ts_manager.cc:194] Registered new tserver with Master: 64737a3a2db540ea946db4bfcaf6f925 (127.22.121.1:33511)
I20260811 06:21:48.255441 24755 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.1:33159
W20260811 06:21:48.597178 25101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:48.597420 25101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:48.597499 25101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:48.604406 25101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:48.604491 25101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:48.604590 25101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:48.613168 25101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:48.614205 25101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:48.615839 25101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:48.628425 25108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:48.628476 25115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:48.628515 25111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:48.629390 25101 server_base.cc:1034] running on GCE node
I20260811 06:21:48.629974 25101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:48.630867 25101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:48.632094 25101 hybrid_clock.cc:648] HybridClock initialized: now 1786429308632026 us; error 49 us; skew 500 ppm
I20260811 06:21:48.632400 25101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:48.634490 25101 webserver.cc:460] Webserver started at http://127.22.121.2:37501/ using document root <none> and password file <none>
I20260811 06:21:48.635111 25101 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:48.635197 25101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:48.635517 25101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:48.638134 25101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "3a5e0f23aa134cb389236067c9d63c65"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "ac686cdb16a1eb2510d1d361fb326bb1"
server_key_iv: "9183e3810d77f4ff29db83595422b908"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.638851 25101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "3a5e0f23aa134cb389236067c9d63c65"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "ac686cdb16a1eb2510d1d361fb326bb1"
server_key_iv: "9183e3810d77f4ff29db83595422b908"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.643810 25101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:21:48.646950 25125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.648170 25101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:48.648308 25101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "3a5e0f23aa134cb389236067c9d63c65"
format_stamp: "Formatted at 2026-08-11 06:21:48 on dist-test-slave-gjrz"
server_key: "ac686cdb16a1eb2510d1d361fb326bb1"
server_key_iv: "9183e3810d77f4ff29db83595422b908"
server_key_version: "encryptionkey@0"
I20260811 06:21:48.648494 25101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:48.673161 25101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:48.673853 25101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:48.674117 25101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:48.674954 25101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:48.676295 25101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:48.676380 25101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.676481 25101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:48.676554 25101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:48.715059 25101 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:37657
I20260811 06:21:48.715150 25317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:37657 every 8 connection(s)
I20260811 06:21:48.716287 25101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:48.724967 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25101
I20260811 06:21:48.725152 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:48.725706 23012 external_mini_cluster.cc:1428] Setting key 864246f13c8bc10f3afbf94bd118419b
I20260811 06:21:48.728690 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:48.718587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:50463 (local address 127.22.121.62:46227)
0811 06:21:48.718767 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:48.718771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:48.720067 (+ 1296us) server_negotiation.cc:407] Connection header received
0811 06:21:48.721794 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:48.721800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:48.721915 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:48.722008 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:48.723027 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.723998 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.726044 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.726536 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.726639 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:48.728041 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:48.728045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:48.728048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:48.728065 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:48.728098 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:48.728109 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:48.728178 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:48.728399 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 06:21:48.728471 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:21:48.728683 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--builtin_ntp_servers=127.22.121.20:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:48.729984 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:48.719064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46227 (local address 127.22.121.2:50463)
0811 06:21:48.719915 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:48.719949 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:48.721534 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:48.722233 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:48.722241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:48.722335 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:48.722861 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:48.722877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.724161 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.724165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:48.725889 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:48.725903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:48.726747 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:48.726753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:48.726968 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:48.727669 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:48.727672 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:48.727721 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:48.727827 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:48.728201 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:48.728212 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:48.728220 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:21:48.728404 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 06:21:48.728772 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":269,"threads_started":1}
I20260811 06:21:48.731204 25319 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46227
I20260811 06:21:48.731484 25319 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:48.732156 25319 heartbeater.cc:511] Master 127.22.121.62:46227 requested a full tablet report, sending...
I20260811 06:21:48.733471 24755 ts_manager.cc:194] Registered new tserver with Master: 3a5e0f23aa134cb389236067c9d63c65 (127.22.121.2:37657)
I20260811 06:21:48.734248 24755 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.2:50463
W20260811 06:21:49.045097 25324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:49.045336 25324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:49.045429 25324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:49.052311 25324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:49.052392 25324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:49.052525 25324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:49.061108 25324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:49.062201 25324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:49.063890 25324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:49.076097 25335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:49.076057 25337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:49.076128 25342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:49.076824 25324 server_base.cc:1034] running on GCE node
I20260811 06:21:49.077346 25324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:49.078126 25324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:49.079329 25324 hybrid_clock.cc:648] HybridClock initialized: now 1786429309079275 us; error 40 us; skew 500 ppm
I20260811 06:21:49.079586 25324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:49.081637 25324 webserver.cc:460] Webserver started at http://127.22.121.3:37257/ using document root <none> and password file <none>
I20260811 06:21:49.082247 25324 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:49.082329 25324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:49.082566 25324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:49.085151 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "406a5bca3a6244c6a392b2608e678ca4"
format_stamp: "Formatted at 2026-08-11 06:21:49 on dist-test-slave-gjrz"
server_key: "b99e2382b5abfb2ad4e9df55d34eee35"
server_key_iv: "594f914eb3ff49cced151062a03cad3d"
server_key_version: "encryptionkey@0"
I20260811 06:21:49.085850 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "406a5bca3a6244c6a392b2608e678ca4"
format_stamp: "Formatted at 2026-08-11 06:21:49 on dist-test-slave-gjrz"
server_key: "b99e2382b5abfb2ad4e9df55d34eee35"
server_key_iv: "594f914eb3ff49cced151062a03cad3d"
server_key_version: "encryptionkey@0"
I20260811 06:21:49.090328 25324 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 06:21:49.093406 25350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:49.094581 25324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:21:49.094704 25324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "406a5bca3a6244c6a392b2608e678ca4"
format_stamp: "Formatted at 2026-08-11 06:21:49 on dist-test-slave-gjrz"
server_key: "b99e2382b5abfb2ad4e9df55d34eee35"
server_key_iv: "594f914eb3ff49cced151062a03cad3d"
server_key_version: "encryptionkey@0"
I20260811 06:21:49.094866 25324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:49.120998 25324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:49.121711 25324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:49.121992 25324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:49.122861 25324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:49.124228 25324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:49.124315 25324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:49.124410 25324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:49.124485 25324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:49.154031 25324 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:33539
I20260811 06:21:49.154090 25537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:33539 every 8 connection(s)
I20260811 06:21:49.155180 25324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:49.165270 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25324
I20260811 06:21:49.165441 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:49.165885 23012 external_mini_cluster.cc:1428] Setting key 93b409a89f81d100fec3f57ff964c41f
I20260811 06:21:49.166791 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.157548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:36575 (local address 127.22.121.62:46227)
0811 06:21:49.157766 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.157772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.159050 (+ 1278us) server_negotiation.cc:407] Connection header received
0811 06:21:49.160785 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.160789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.160856 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.160946 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:49.161872 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.162645 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.164285 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.164693 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.164785 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.165979 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:49.165986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:49.165990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:49.166016 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:49.166071 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:49.166089 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:49.166270 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.166441 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.166522 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:21:49.167850 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.158054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:46227 (local address 127.22.121.3:36575)
0811 06:21:49.158895 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.158933 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.160514 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.161121 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.161129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.161215 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:49.161742 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.161758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.162765 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.162768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.164151 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.164164 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.164849 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.164854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.165048 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.165673 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:49.165675 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:49.165714 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:49.165782 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:49.166255 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:49.166263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:49.166270 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.166406 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.166751 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":251,"threads_started":1}
I20260811 06:21:49.169126 25539 heartbeater.cc:347] Connected to a master server at 127.22.121.62:46227
I20260811 06:21:49.169373 25539 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:49.170049 25539 heartbeater.cc:511] Master 127.22.121.62:46227 requested a full tablet report, sending...
I20260811 06:21:49.171298 24754 ts_manager.cc:194] Registered new tserver with Master: 406a5bca3a6244c6a392b2608e678ca4 (127.22.121.3:33539)
I20260811 06:21:49.172194 24754 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.3:36575
I20260811 06:21:49.180843 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:49.189810 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.184627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.22.121.62:46227)
0811 06:21:49.184766 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.184770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.185232 (+ 462us) server_negotiation.cc:407] Connection header received
0811 06:21:49.185351 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.185354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.185413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.185488 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:49.185902 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.186577 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.187863 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.188273 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.188364 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.189264 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:49.189269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:49.189272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:49.189287 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:49.189315 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:49.189327 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:49.189438 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.189555 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.189600 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:21:49.193281 24754 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:49.200201 24754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:49.222359 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.211563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53800)
0811 06:21:49.212467 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.212511 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.212698 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.213970 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.213978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.214010 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:49.214786 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.214803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.216759 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.216765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.219148 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.219165 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.221408 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.221415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.221656 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.221707 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.221829 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.222027 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":298,"threads_started":1}
I20260811 06:21:49.223407 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.211630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.22.121.1:33511)
0811 06:21:49.213111 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.213120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.213149 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:21:49.213247 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.213255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.213511 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.213716 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:49.214989 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.216585 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.219430 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.220911 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.221038 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.221280 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.222883 (+ 1603us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.223068 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":1436,"threads_started":1}
I20260811 06:21:49.229745 24981 tablet_service.cc:1467] Processing CreateTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b (DEFAULT_TABLE table=test-table [id=81a9717c1d5a4e4385d12e76e1d54447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:49.231245 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.212591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:33539 (local address 127.0.0.1:48484)
0811 06:21:49.222614 (+ 10023us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.222633 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.222727 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.224667 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.224672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.224700 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:49.224894 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.224904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.227013 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.227022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.228838 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.228855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.230460 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.230466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.230631 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.230663 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.230740 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.230876 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9923,"mutex_wait_us":38,"thread_start_us":13125,"threads_started":1}
I20260811 06:21:49.231473 24981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aca67ee2ce24b16a79de8bd1e34787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:49.231580 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.213153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.22.121.3:33539)
0811 06:21:49.224022 (+ 10869us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.224030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.224065 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:21:49.224167 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.224176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.224434 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.225075 (+ 641us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:49.225371 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.226839 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.228986 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.230284 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.230412 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.230965 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.231052 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.231203 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10702,"thread_start_us":10557,"threads_started":1}
I20260811 06:21:49.235237 25434 tablet_service.cc:1467] Processing CreateTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b (DEFAULT_TABLE table=test-table [id=81a9717c1d5a4e4385d12e76e1d54447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:49.236594 25434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aca67ee2ce24b16a79de8bd1e34787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:49.238238 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.212891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:37657 (local address 127.0.0.1:52898)
0811 06:21:49.225984 (+ 13093us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.226000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.226127 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.226634 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.226638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.226661 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:49.227109 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.227125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.228905 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.228910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.230329 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.230342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.237490 (+ 7148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.237496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.237681 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.237714 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.237798 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.237918 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12960,"mutex_wait_us":37,"thread_start_us":13111,"threads_started":1}
I20260811 06:21:49.238968 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.213321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.22.121.2:37657)
0811 06:21:49.225702 (+ 12381us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.225709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.226192 (+ 483us) server_negotiation.cc:407] Connection header received
0811 06:21:49.226277 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.226282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.226468 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.226633 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:49.227319 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.228763 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.235646 (+ 6883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.237019 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.237154 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.237365 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.238459 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.238642 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12221,"thread_start_us":12105,"threads_started":1}
I20260811 06:21:49.242300 25210 tablet_service.cc:1467] Processing CreateTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b (DEFAULT_TABLE table=test-table [id=81a9717c1d5a4e4385d12e76e1d54447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:49.243554 25210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aca67ee2ce24b16a79de8bd1e34787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:49.245690 25578 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Bootstrap starting.
I20260811 06:21:49.249322 25578 tablet_bootstrap.cc:654] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:49.249861 25579 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Bootstrap starting.
I20260811 06:21:49.251505 25578 log.cc:824] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:49.253291 25579 tablet_bootstrap.cc:654] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:49.253939 25578 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: No bootstrap required, opened a new log
I20260811 06:21:49.254174 25578 ts_tablet_manager.cc:1397] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:21:49.254422 25579 log.cc:824] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:49.257110 25579 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: No bootstrap required, opened a new log
I20260811 06:21:49.257326 25579 ts_tablet_manager.cc:1397] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 06:21:49.258050 25092 heartbeater.cc:503] Master 127.22.121.62:46227 was elected leader, sending a full tablet report...
I20260811 06:21:49.262514 25578 raft_consensus.cc:348] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.262854 25578 raft_consensus.cc:374] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:49.262936 25578 raft_consensus.cc:729] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64737a3a2db540ea946db4bfcaf6f925, State: Initialized, Role: FOLLOWER
I20260811 06:21:49.263454 25578 consensus_queue.cc:260] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.265022 25578 ts_tablet_manager.cc:1428] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 06:21:49.265225 25579 raft_consensus.cc:348] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.265525 25579 raft_consensus.cc:374] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:49.265612 25579 raft_consensus.cc:729] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a5bca3a6244c6a392b2608e678ca4, State: Initialized, Role: FOLLOWER
I20260811 06:21:49.266150 25579 consensus_queue.cc:260] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.266760 25583 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Bootstrap starting.
I20260811 06:21:49.267695 25539 heartbeater.cc:503] Master 127.22.121.62:46227 was elected leader, sending a full tablet report...
I20260811 06:21:49.268690 25579 ts_tablet_manager.cc:1428] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 06:21:49.268692 25587 raft_consensus.cc:482] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:49.268971 25587 raft_consensus.cc:504] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.269790 25583 tablet_bootstrap.cc:654] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:49.270871 25583 log.cc:824] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:49.272889 25587 leader_election.cc:288] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 406a5bca3a6244c6a392b2608e678ca4 (127.22.121.3:33539), 3a5e0f23aa134cb389236067c9d63c65 (127.22.121.2:37657)
I20260811 06:21:49.279637 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.271112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:53419 (local address 127.22.121.3:33539)
0811 06:21:49.271295 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.271301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.271643 (+ 342us) server_negotiation.cc:407] Connection header received
0811 06:21:49.271816 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.271822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.271899 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.271989 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:49.272832 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.274037 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.277172 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.278439 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.278577 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.278649 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.279211 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.279367 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":36}
I20260811 06:21:49.280285 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.270667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:33539 (local address 127.22.121.1:53419)
0811 06:21:49.271530 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.271555 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.271688 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.272149 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.272154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.272191 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:49.272664 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.272678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.274619 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.274625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.276929 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.276946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.278825 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.278831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.279021 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.279060 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.279855 (+ 795us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.279999 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":643,"threads_started":1}
I20260811 06:21:49.281168 25467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aca67ee2ce24b16a79de8bd1e34787b" candidate_uuid: "64737a3a2db540ea946db4bfcaf6f925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a5bca3a6244c6a392b2608e678ca4" is_pre_election: true
I20260811 06:21:49.281657 25467 raft_consensus.cc:2454] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64737a3a2db540ea946db4bfcaf6f925 in term 0.
I20260811 06:21:49.282356 24906 leader_election.cc:302] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a5bca3a6244c6a392b2608e678ca4, 64737a3a2db540ea946db4bfcaf6f925; no voters:
I20260811 06:21:49.282775 25587 raft_consensus.cc:2781] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:49.282817 25583 tablet_bootstrap.cc:492] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: No bootstrap required, opened a new log
I20260811 06:21:49.282967 25587 raft_consensus.cc:482] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:49.283071 25583 ts_tablet_manager.cc:1397] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.009s
I20260811 06:21:49.283142 25587 raft_consensus.cc:3037] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:49.285396 25587 raft_consensus.cc:504] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.286116 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.271336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:55739 (local address 127.22.121.2:37657)
0811 06:21:49.271514 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.271519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.279859 (+ 8340us) server_negotiation.cc:407] Connection header received
0811 06:21:49.279977 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.279983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.280080 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.280177 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:49.280781 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.281499 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.283861 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.285142 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.285297 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.285369 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.285734 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.285861 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:21:49.286140 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.270712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:37657 (local address 127.22.121.1:55739)
0811 06:21:49.279736 (+ 9024us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.279754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:49.279865 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:49.280263 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:49.280268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:49.280297 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:49.280609 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.280622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.281625 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.281631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.283671 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:49.283690 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.285349 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.285359 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:49.285534 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.285567 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:49.285703 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:21:49.285819 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8903,"thread_start_us":8727,"threads_started":1}
I20260811 06:21:49.286350 25467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aca67ee2ce24b16a79de8bd1e34787b" candidate_uuid: "64737a3a2db540ea946db4bfcaf6f925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a5bca3a6244c6a392b2608e678ca4"
I20260811 06:21:49.286530 25467 raft_consensus.cc:3037] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:49.287169 25587 leader_election.cc:288] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [CANDIDATE]: Term 1 election: Requested vote from peers 406a5bca3a6244c6a392b2608e678ca4 (127.22.121.3:33539), 3a5e0f23aa134cb389236067c9d63c65 (127.22.121.2:37657)
I20260811 06:21:49.287738 25247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aca67ee2ce24b16a79de8bd1e34787b" candidate_uuid: "64737a3a2db540ea946db4bfcaf6f925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5e0f23aa134cb389236067c9d63c65"
I20260811 06:21:49.287468 25248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aca67ee2ce24b16a79de8bd1e34787b" candidate_uuid: "64737a3a2db540ea946db4bfcaf6f925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5e0f23aa134cb389236067c9d63c65" is_pre_election: true
I20260811 06:21:49.289354 25467 raft_consensus.cc:2454] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64737a3a2db540ea946db4bfcaf6f925 in term 1.
W20260811 06:21:49.289626 24907 leader_election.cc:341] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5e0f23aa134cb389236067c9d63c65 (127.22.121.2:37657): Illegal state: must be running to vote when last-logged opid is not known
I20260811 06:21:49.289827 24906 leader_election.cc:302] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 406a5bca3a6244c6a392b2608e678ca4, 64737a3a2db540ea946db4bfcaf6f925; no voters: 3a5e0f23aa134cb389236067c9d63c65
I20260811 06:21:49.290256 25587 raft_consensus.cc:2781] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:49.290849 25587 raft_consensus.cc:686] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 1 LEADER]: Becoming Leader. State: Replica: 64737a3a2db540ea946db4bfcaf6f925, State: Running, Role: LEADER
I20260811 06:21:49.291332 25587 consensus_queue.cc:237] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.292388 25583 raft_consensus.cc:348] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.292670 25583 raft_consensus.cc:374] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:49.292760 25583 raft_consensus.cc:729] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5e0f23aa134cb389236067c9d63c65, State: Initialized, Role: FOLLOWER
I20260811 06:21:49.293320 25583 consensus_queue.cc:260] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } }
I20260811 06:21:49.294359 25319 heartbeater.cc:503] Master 127.22.121.62:46227 was elected leader, sending a full tablet report...
I20260811 06:21:49.295058 25583 ts_tablet_manager.cc:1428] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260811 06:21:49.295646 24754 catalog_manager.cc:5495] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64737a3a2db540ea946db4bfcaf6f925 (127.22.121.1). New cstate: current_term: 1 leader_uuid: "64737a3a2db540ea946db4bfcaf6f925" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64737a3a2db540ea946db4bfcaf6f925" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 33511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 } health_report { overall_health: UNKNOWN } } }
I20260811 06:21:49.320523 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.315944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.22.121.1:33511)
0811 06:21:49.316100 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.316104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.316119 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:21:49.316252 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.316256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.316342 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.316464 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:49.316779 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.317429 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.318726 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.319126 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.319215 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.319653 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:49.320072 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:49.320189 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.320287 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.320320 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:21:49.327696 25247 raft_consensus.cc:3037] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:49.327929 25467 raft_consensus.cc:1261] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 1 FOLLOWER]: Refusing update from remote peer 64737a3a2db540ea946db4bfcaf6f925: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:49.328680 25597 consensus_queue.cc:1035] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a5bca3a6244c6a392b2608e678ca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:49.330603 25247 raft_consensus.cc:1261] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 1 FOLLOWER]: Refusing update from remote peer 64737a3a2db540ea946db4bfcaf6f925: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:49.331328 25597 consensus_queue.cc:1035] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5e0f23aa134cb389236067c9d63c65" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 37657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:49.338297 25607 mvcc.cc:205] Tried to move back new op lower bound from 7317214450997829632 to 7317214450860625920. Current Snapshot: MvccSnapshot[applied={T|T < 7317214450997829632}]
I20260811 06:21:49.351138 24755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:49.351413 24755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:49.355669 24755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c10a5b81ebc642f09c93d2a96a9c8bde: Sending DeleteTablet for 3 replicas of tablet 0aca67ee2ce24b16a79de8bd1e34787b
I20260811 06:21:49.357409 25434 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:49 UTC) from {username='slave'} at 127.0.0.1:48484
I20260811 06:21:49.357420 25210 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:49 UTC) from {username='slave'} at 127.0.0.1:52898
I20260811 06:21:49.358273 24981 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aca67ee2ce24b16a79de8bd1e34787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:49 UTC) from {username='slave'} at 127.0.0.1:53800
I20260811 06:21:49.360203 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24875
I20260811 06:21:49.371683 25627 tablet_replica.cc:330] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: stopping tablet replica
I20260811 06:21:49.372008 25627 raft_consensus.cc:2229] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:49.371218 25621 tablet_replica.cc:330] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: stopping tablet replica
I20260811 06:21:49.375425 25627 raft_consensus.cc:2258] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:49.375749 25621 raft_consensus.cc:2229] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:49.376194 25621 raft_consensus.cc:2258] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:49.377300 25620 tablet_replica.cc:330] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: stopping tablet replica
I20260811 06:21:49.377602 25627 ts_tablet_manager.cc:1905] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:49.380846 25620 raft_consensus.cc:2229] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:49.381000 25621 ts_tablet_manager.cc:1905] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:49.381721 25620 raft_consensus.cc:2258] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:49.382690 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.361435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.22.121.1:33511)
0811 06:21:49.361610 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.361616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.374471 (+ 12855us) server_negotiation.cc:407] Connection header received
0811 06:21:49.374634 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.374640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.374730 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.374854 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:49.375651 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.376810 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.378820 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.379502 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.379607 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.381010 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:49.381032 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:49.381046 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:49.381726 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:49.381787 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:49.381812 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:49.381882 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.382218 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.382323 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:21:49.383920 25082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:49.385821 25620 ts_tablet_manager.cc:1905] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:49.387751 25621 ts_tablet_manager.cc:1918] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:49.387910 25621 log.cc:1196] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/0aca67ee2ce24b16a79de8bd1e34787b
I20260811 06:21:49.388418 25621 ts_tablet_manager.cc:1939] T 0aca67ee2ce24b16a79de8bd1e34787b P 406a5bca3a6244c6a392b2608e678ca4: Deleting consensus metadata
I20260811 06:21:49.390128 24729 catalog_manager.cc:4841] TS 406a5bca3a6244c6a392b2608e678ca4 (127.22.121.3:33539): tablet 0aca67ee2ce24b16a79de8bd1e34787b (table test-table [id=81a9717c1d5a4e4385d12e76e1d54447]) successfully deleted
I20260811 06:21:49.390314 25620 ts_tablet_manager.cc:1918] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:49.390457 25620 log.cc:1196] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/0aca67ee2ce24b16a79de8bd1e34787b
I20260811 06:21:49.390930 25620 ts_tablet_manager.cc:1939] T 0aca67ee2ce24b16a79de8bd1e34787b P 3a5e0f23aa134cb389236067c9d63c65: Deleting consensus metadata
I20260811 06:21:49.392385 24730 catalog_manager.cc:4841] TS 3a5e0f23aa134cb389236067c9d63c65 (127.22.121.2:37657): tablet 0aca67ee2ce24b16a79de8bd1e34787b (table test-table [id=81a9717c1d5a4e4385d12e76e1d54447]) successfully deleted
I20260811 06:21:49.481145 25627 ts_tablet_manager.cc:1918] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:49.481518 25627 log.cc:1196] T 0aca67ee2ce24b16a79de8bd1e34787b P 64737a3a2db540ea946db4bfcaf6f925: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/0aca67ee2ce24b16a79de8bd1e34787b
I20260811 06:21:49.481783 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24875
W20260811 06:21:49.510263 24729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.121.1:33511 (error 108)
I20260811 06:21:49.511482 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25101
I20260811 06:21:49.512027 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.511594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53824)
0811 06:21:49.511753 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.511865 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 06:21:49.512318 24729 catalog_manager.cc:4584] TS 64737a3a2db540ea946db4bfcaf6f925 (127.22.121.1:33511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0aca67ee2ce24b16a79de8bd1e34787b: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
I20260811 06:21:49.518348 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.512226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.22.121.2:37657)
0811 06:21:49.512380 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.512386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.512509 (+ 123us) server_negotiation.cc:407] Connection header received
0811 06:21:49.512696 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.512700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.512800 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.512891 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:49.513311 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.514052 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.515427 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.515836 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.515947 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.516997 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:49.517006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:49.517025 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:49.517618 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:49.517666 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:49.517692 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:49.517793 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.517936 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.518052 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 06:21:49.518970 25310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:49.565743 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.565333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53826)
0811 06:21:49.565517 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.565595 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:21:49.566696 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.566369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53830)
0811 06:21:49.566508 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.566572 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:21:49.604251 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25101
I20260811 06:21:49.624228 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25324
I20260811 06:21:49.626828 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.626420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53838)
0811 06:21:49.626607 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.626681 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:21:49.627754 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.627430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:53854)
0811 06:21:49.627569 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.627642 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 06:21:49.633643 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.624917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.22.121.3:33539)
0811 06:21:49.625088 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:49.625094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:49.625113 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:21:49.625306 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:49.625310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:49.625411 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:49.625498 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:49.625912 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.626643 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.628140 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:49.628559 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:49.628670 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:49.632327 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:49.632337 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:49.632353 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:49.632943 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:49.632998 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:49.633017 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:49.633122 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:49.633251 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:21:49.633359 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:21:49.634263 25533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:49.708034 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.707640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:50396)
0811 06:21:49.707816 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.707893 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 06:21:49.709002 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:49.708662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:33511 (local address 127.0.0.1:50406)
0811 06:21:49.708834 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:49.708902 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.1:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 06:21:49.735365 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25324
I20260811 06:21:49.757040 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 24701
I20260811 06:21:49.758062 24817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:49.861407 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 24701
2026-08-11T06:21:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:21:49 dist-test-slave-gjrz krb5kdc[25646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:21:49 dist-test-slave-gjrz krb5kdc[25646](info): set up 2 sockets
Aug 11 06:21:49 dist-test-slave-gjrz krb5kdc[25646](info): commencing operation
krb5kdc: starting...
W20260811 06:21:51.097151 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:21:51 dist-test-slave-gjrz krb5kdc[25646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:21:51.332407 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:44137
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:44137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:51.653714 25670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:51.653966 25670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:51.654096 25670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:51.660984 25670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:51.661077 25670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:51.661147 25670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:51.661217 25670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:51.661274 25670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:51.670347 25670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:44137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:44137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:51.671456 25670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:51.672629 25670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:51.683426 25686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:51.683537 25681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:51.683563 25684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:51.684479 25670 server_base.cc:1034] running on GCE node
I20260811 06:21:51.685141 25670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:51.686353 25670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:51.687548 25670 hybrid_clock.cc:648] HybridClock initialized: now 1786429311687502 us; error 37 us; skew 500 ppm
I20260811 06:21:51.687850 25670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:51 dist-test-slave-gjrz krb5kdc[25646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:51.690816 25670 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:21:51.692203 25670 webserver.cc:460] Webserver started at http://127.22.121.62:35615/ using document root <none> and password file <none>
I20260811 06:21:51.692837 25670 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:51.692957 25670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:51.693229 25670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:51.696044 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "a2bc8314145c4d34adc54b886915abf7"
format_stamp: "Formatted at 2026-08-11 06:21:51 on dist-test-slave-gjrz"
server_key: "fd99b811777cbedfff74e25cddc320b8"
server_key_iv: "b60b03070cbafc52ba6a7526ba084af8"
server_key_version: "encryptionkey@0"
I20260811 06:21:51.696748 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "a2bc8314145c4d34adc54b886915abf7"
format_stamp: "Formatted at 2026-08-11 06:21:51 on dist-test-slave-gjrz"
server_key: "fd99b811777cbedfff74e25cddc320b8"
server_key_iv: "b60b03070cbafc52ba6a7526ba084af8"
server_key_version: "encryptionkey@0"
I20260811 06:21:51.701480 25670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:21:51.704744 25695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:51.705900 25670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:21:51.706053 25670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a2bc8314145c4d34adc54b886915abf7"
format_stamp: "Formatted at 2026-08-11 06:21:51 on dist-test-slave-gjrz"
server_key: "fd99b811777cbedfff74e25cddc320b8"
server_key_iv: "b60b03070cbafc52ba6a7526ba084af8"
server_key_version: "encryptionkey@0"
I20260811 06:21:51.706221 25670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:51.718668 25670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:51.722146 25670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:51.722386 25670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:51.739145 25670 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:44137
I20260811 06:21:51.739164 25785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:44137 every 8 connection(s)
I20260811 06:21:51.740350 25670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:51.743706 25786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:51.750595 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25670
I20260811 06:21:51.750782 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:51.751338 23012 external_mini_cluster.cc:1428] Setting key d7b3923b5d5694f5d55ec876f7e90a92
I20260811 06:21:51.753856 25786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: Bootstrap starting.
I20260811 06:21:51.756976 25786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:51.758193 25786 log.cc:824] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:51.761116 25786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: No bootstrap required, opened a new log
Aug 11 06:21:51 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:51.769459 25786 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } }
I20260811 06:21:51.769778 25786 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:51.769870 25786 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2bc8314145c4d34adc54b886915abf7, State: Initialized, Role: FOLLOWER
I20260811 06:21:51.770439 25786 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } }
I20260811 06:21:51.770655 25786 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:51.770825 25786 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:51.770987 25786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:51.773262 25786 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } }
I20260811 06:21:51.773483 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:51.753036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.22.121.62:44137)
0811 06:21:51.753731 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:51.753742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:51.753780 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:21:51.755573 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:51.755595 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:51.755918 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:51.756201 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:51.757960 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:51.759314 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:51.761084 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:51.761789 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:51.761924 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:51.766410 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:51.766418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:51.766432 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:51.766469 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:51.769441 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:51.770186 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:51.770192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:51.770201 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:51.770368 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:51.770774 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:51.770778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:51.770780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:51.771399 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:51.771710 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:51.772194 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 06:21:51.772581 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":322,"threads_started":1}
I20260811 06:21:51.773756 25786 leader_election.cc:302] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2bc8314145c4d34adc54b886915abf7; no voters:
I20260811 06:21:51.774360 25786 leader_election.cc:288] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:51.774461 25795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:51.775604 25795 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [term 1 LEADER]: Becoming Leader. State: Replica: a2bc8314145c4d34adc54b886915abf7, State: Running, Role: LEADER
I20260811 06:21:51.776233 25795 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } }
I20260811 06:21:51.776554 25786 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:51.779347 25804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2bc8314145c4d34adc54b886915abf7. Latest consensus state: current_term: 1 leader_uuid: "a2bc8314145c4d34adc54b886915abf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } } }
I20260811 06:21:51.779429 25799 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2bc8314145c4d34adc54b886915abf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bc8314145c4d34adc54b886915abf7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44137 } } }
I20260811 06:21:51.779753 25804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:51.779780 25799 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:51.781265 25809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:51.786176 25809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:51.797317 25809 catalog_manager.cc:1347] Generated new cluster ID: f0ed0e571f34462badac60e23437916e
I20260811 06:21:51.797428 25809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:51.828277 25809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:51.829473 25809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:51.834460 25809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: Generated new TSK 0
I20260811 06:21:51.835245 25809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.121.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
Entry for principal kudu/127.22.121.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab.
I20260811 06:21:51.910315 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:52.239192 25843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:52.239471 25843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:52.239574 25843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:52.246392 25843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:52.246482 25843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:52.246589 25843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:52.255672 25843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.1.keytab
--principal=kudu/127.22.121.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:52.256705 25843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:52.257891 25843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:52.270192 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:52.270241 25856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:52.270166 25857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:52.270752 25843 server_base.cc:1034] running on GCE node
I20260811 06:21:52.271338 25843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:52.272166 25843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:52.273351 25843 hybrid_clock.cc:648] HybridClock initialized: now 1786429312273311 us; error 32 us; skew 500 ppm
I20260811 06:21:52.273625 25843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:52 dist-test-slave-gjrz krb5kdc[25646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:52.276648 25843 init.cc:377] Logged in from keytab as kudu/127.22.121.1@KRBTEST.COM (short username kudu)
I20260811 06:21:52.278044 25843 webserver.cc:460] Webserver started at http://127.22.121.1:38691/ using document root <none> and password file <none>
I20260811 06:21:52.278754 25843 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:52.278869 25843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:52.279170 25843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:52.281904 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "964420f356e14ca7a27388477481aabf"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "7d1a0b6482ce2bd5bd34b8c1377ed5b6"
server_key_iv: "23ac182ec453916c3a97218ecba3e141"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.282625 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "964420f356e14ca7a27388477481aabf"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "7d1a0b6482ce2bd5bd34b8c1377ed5b6"
server_key_iv: "23ac182ec453916c3a97218ecba3e141"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.287328 25843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 06:21:52.290736 25869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.291934 25843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:21:52.292088 25843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "964420f356e14ca7a27388477481aabf"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "7d1a0b6482ce2bd5bd34b8c1377ed5b6"
server_key_iv: "23ac182ec453916c3a97218ecba3e141"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.292253 25843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:52.312636 25843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:52.316246 25843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:52.316478 25843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:52.317282 25843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:52.318415 25843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:52.318485 25843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.318580 25843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:52.318627 25843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.348374 25843 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:45793
I20260811 06:21:52.348433 26084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:45793 every 8 connection(s)
I20260811 06:21:52.349498 25843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:52.355669 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 25843
I20260811 06:21:52.355834 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:52.356231 23012 external_mini_cluster.cc:1428] Setting key 5730214ea8e401ff971e92eb1d54ff9c
Aug 11 06:21:52 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.1@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:52.367779 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:52.351790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43723 (local address 127.22.121.62:44137)
0811 06:21:52.352205 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:52.352210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:52.353006 (+ 796us) server_negotiation.cc:407] Connection header received
0811 06:21:52.354774 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:52.354780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:52.354855 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:52.354951 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:52.356463 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.357294 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.358807 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.359203 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.359333 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:52.362546 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:52.362551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:52.362555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:52.362586 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:52.364929 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:52.365611 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:52.365617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:52.365620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:52.365741 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:52.366127 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:52.366133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:52.366136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:52.366395 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:52.366534 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:52.367188 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 06:21:52.367433 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":231,"threads_started":1}
I20260811 06:21:52.368618 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:52.352207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:44137 (local address 127.22.121.1:43723)
0811 06:21:52.352845 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:52.352891 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:52.354518 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:52.355185 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:52.355193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:52.355699 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:52.356260 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:52.356278 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.357469 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.357473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:52.358671 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:52.358683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.359437 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.359444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:52.359667 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:52.360252 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:52.360278 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:52.362343 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:52.365056 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:52.365064 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:52.365078 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:52.365489 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:52.365841 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:52.365847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:52.365850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:52.366021 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:52.366560 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:52.366568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:52.366838 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 06:21:52.367057 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 06:21:52.367605 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":239,"threads_started":1}
I20260811 06:21:52.369976 26085 heartbeater.cc:347] Connected to a master server at 127.22.121.62:44137
I20260811 06:21:52.370266 26085 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:52.371011 26085 heartbeater.cc:511] Master 127.22.121.62:44137 requested a full tablet report, sending...
I20260811 06:21:52.373296 25728 ts_manager.cc:194] Registered new tserver with Master: 964420f356e14ca7a27388477481aabf (127.22.121.1:45793)
I20260811 06:21:52.375228 25728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.1@KRBTEST.COM'} at 127.22.121.1:43723
WARNING: no policy specified for kudu/127.22.121.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
Entry for principal kudu/127.22.121.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab.
I20260811 06:21:52.414283 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:52.721787 26092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:52.722041 26092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:52.722138 26092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:52.729238 26092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:52.729333 26092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:52.729445 26092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:52.738330 26092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.2.keytab
--principal=kudu/127.22.121.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:52.739477 26092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:52.740635 26092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:52.752666 26100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:52.752689 26101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:52.752852 26104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:52.755009 26092 server_base.cc:1034] running on GCE node
I20260811 06:21:52.755573 26092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:52.756358 26092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:52.757555 26092 hybrid_clock.cc:648] HybridClock initialized: now 1786429312757509 us; error 37 us; skew 500 ppm
I20260811 06:21:52.757825 26092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:52 dist-test-slave-gjrz krb5kdc[25646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:52.760692 26092 init.cc:377] Logged in from keytab as kudu/127.22.121.2@KRBTEST.COM (short username kudu)
I20260811 06:21:52.762068 26092 webserver.cc:460] Webserver started at http://127.22.121.2:33253/ using document root <none> and password file <none>
I20260811 06:21:52.762691 26092 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:52.762806 26092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:52.763088 26092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:52.765743 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "0e36c185a9e3429cab9cdaaa74084dd6"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "9ab1e73ca2b330d4a57def3437175a8f"
server_key_iv: "178588e4e2b3c9de6c5f4885bc710289"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.766448 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "0e36c185a9e3429cab9cdaaa74084dd6"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "9ab1e73ca2b330d4a57def3437175a8f"
server_key_iv: "178588e4e2b3c9de6c5f4885bc710289"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.771066 26092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 06:21:52.774138 26120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.775290 26092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:52.775437 26092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "0e36c185a9e3429cab9cdaaa74084dd6"
format_stamp: "Formatted at 2026-08-11 06:21:52 on dist-test-slave-gjrz"
server_key: "9ab1e73ca2b330d4a57def3437175a8f"
server_key_iv: "178588e4e2b3c9de6c5f4885bc710289"
server_key_version: "encryptionkey@0"
I20260811 06:21:52.775609 26092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:52.786489 26092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:52.790295 26092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:52.790536 26092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:52.791368 26092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:52.792541 26092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:52.792615 26092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.792728 26092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:52.792783 26092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:52.824105 26092 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:33001
I20260811 06:21:52.824162 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:33001 every 8 connection(s)
I20260811 06:21:52.825248 26092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:52.834671 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26092
I20260811 06:21:52.834851 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:52.835354 23012 external_mini_cluster.cc:1428] Setting key b09bcd1688991afe8f57c51e1d3d70a5
Aug 11 06:21:52 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.2@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:52.843417 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:52.827515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:39599 (local address 127.22.121.62:44137)
0811 06:21:52.827723 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:52.827729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:52.828744 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 06:21:52.830456 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:52.830460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:52.830528 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:52.830612 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:52.831990 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.832710 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.834138 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.834549 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.834659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:52.838222 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:52.838227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:52.838230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:52.838261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:52.840630 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:52.841273 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:52.841277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:52.841279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:52.841403 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:52.841789 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:52.841792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:52.841794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:52.842080 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:52.842209 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:52.842835 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 06:21:52.843065 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:21:52.844195 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:52.827931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:44137 (local address 127.22.121.2:39599)
0811 06:21:52.828593 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:52.828629 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:52.830213 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:52.830790 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:52.830798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:52.831257 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:52.831845 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:52.831862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.832824 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.832828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:52.834024 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:52.834034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:52.834758 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:52.834765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:52.835035 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:52.835657 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:52.835687 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:52.838023 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:52.840753 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:52.840761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:52.840775 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:52.841157 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:52.841503 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:52.841508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:52.841512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:52.841681 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:52.842203 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:52.842211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:52.842498 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 06:21:52.842738 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 06:21:52.843189 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":251,"threads_started":1}
I20260811 06:21:52.845512 26335 heartbeater.cc:347] Connected to a master server at 127.22.121.62:44137
I20260811 06:21:52.845777 26335 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:52.846386 26335 heartbeater.cc:511] Master 127.22.121.62:44137 requested a full tablet report, sending...
I20260811 06:21:52.847700 25728 ts_manager.cc:194] Registered new tserver with Master: 0e36c185a9e3429cab9cdaaa74084dd6 (127.22.121.2:33001)
I20260811 06:21:52.848455 25728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.2@KRBTEST.COM'} at 127.22.121.2:39599
WARNING: no policy specified for kudu/127.22.121.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
Entry for principal kudu/127.22.121.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab.
I20260811 06:21:52.891394 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:53.203855 26342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:53.204113 26342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:53.204206 26342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:53.211091 26342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:53.211179 26342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:53.211298 26342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:53.220258 26342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.3.keytab
--principal=kudu/127.22.121.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:44137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:53.221300 26342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:53.222476 26342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:53.234611 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:53.234661 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:53.234814 26358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:53.235699 26342 server_base.cc:1034] running on GCE node
I20260811 06:21:53.236260 26342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:53.237123 26342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:53.238329 26342 hybrid_clock.cc:648] HybridClock initialized: now 1786429313238283 us; error 37 us; skew 500 ppm
I20260811 06:21:53.238631 26342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:53 dist-test-slave-gjrz krb5kdc[25646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429313, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:53.241714 26342 init.cc:377] Logged in from keytab as kudu/127.22.121.3@KRBTEST.COM (short username kudu)
I20260811 06:21:53.243010 26342 webserver.cc:460] Webserver started at http://127.22.121.3:41207/ using document root <none> and password file <none>
I20260811 06:21:53.243654 26342 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:53.243753 26342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:53.244015 26342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:53.246716 26342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "684441c3f2d141e89cf35bdea0158b55"
format_stamp: "Formatted at 2026-08-11 06:21:53 on dist-test-slave-gjrz"
server_key: "7dfb6740d91ebfddabdae885bdbf1b6d"
server_key_iv: "1c7763dc8a5dcdd8581668d5fdb2af72"
server_key_version: "encryptionkey@0"
I20260811 06:21:53.247463 26342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "684441c3f2d141e89cf35bdea0158b55"
format_stamp: "Formatted at 2026-08-11 06:21:53 on dist-test-slave-gjrz"
server_key: "7dfb6740d91ebfddabdae885bdbf1b6d"
server_key_iv: "1c7763dc8a5dcdd8581668d5fdb2af72"
server_key_version: "encryptionkey@0"
I20260811 06:21:53.252241 26342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:21:53.255440 26365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:53.256633 26342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 06:21:53.256774 26342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "684441c3f2d141e89cf35bdea0158b55"
format_stamp: "Formatted at 2026-08-11 06:21:53 on dist-test-slave-gjrz"
server_key: "7dfb6740d91ebfddabdae885bdbf1b6d"
server_key_iv: "1c7763dc8a5dcdd8581668d5fdb2af72"
server_key_version: "encryptionkey@0"
I20260811 06:21:53.256947 26342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:53.280242 26342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:53.283855 26342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:53.284103 26342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:53.284932 26342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:53.286120 26342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:53.286190 26342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:53.286294 26342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:53.286342 26342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:53.314937 26342 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:33989
I20260811 06:21:53.315017 26554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:33989 every 8 connection(s)
I20260811 06:21:53.316181 26342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:53.322274 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26342
I20260811 06:21:53.322448 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:53.322854 23012 external_mini_cluster.cc:1428] Setting key 57d14d6af33495f781f0c2af97953147
Aug 11 06:21:53 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429313, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.3@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:21:53.334839 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.318592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:40897 (local address 127.22.121.62:44137)
0811 06:21:53.318772 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.318777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.320160 (+ 1383us) server_negotiation.cc:407] Connection header received
0811 06:21:53.322022 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.322029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.322119 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.322212 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:53.323919 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.324860 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.326508 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.326905 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.326999 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.330120 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:53.330125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:53.330128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:53.330158 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:53.332465 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.333088 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.333091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.333093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.333216 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.333552 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.333555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.333557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.333799 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:53.333904 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.334442 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.334602 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:21:53.335752 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.319020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:44137 (local address 127.22.121.3:40897)
0811 06:21:53.319967 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.320013 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.321705 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.322484 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.322492 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.323042 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:53.323759 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.323779 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.325010 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.325014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.326382 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.326395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.327088 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.327093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.327329 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.327886 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:21:53.327907 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:53.329928 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:53.332588 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:53.332594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:53.332606 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:53.332991 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:53.333295 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:21:53.333298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:21:53.333300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:21:53.333463 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:21:53.333890 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:53.333895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:53.334161 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.334365 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.334720 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":313,"threads_started":1}
I20260811 06:21:53.337031 26555 heartbeater.cc:347] Connected to a master server at 127.22.121.62:44137
I20260811 06:21:53.337334 26555 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:53.338028 26555 heartbeater.cc:511] Master 127.22.121.62:44137 requested a full tablet report, sending...
I20260811 06:21:53.339334 25727 ts_manager.cc:194] Registered new tserver with Master: 684441c3f2d141e89cf35bdea0158b55 (127.22.121.3:33989)
I20260811 06:21:53.340131 25727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.121.3@KRBTEST.COM'} at 127.22.121.3:40897
I20260811 06:21:53.349819 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:53.366187 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.354660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.22.121.62:44137)
0811 06:21:53.354815 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.354821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.355211 (+ 390us) server_negotiation.cc:407] Connection header received
0811 06:21:53.355387 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.355393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.355458 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.355535 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:53.356899 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.357614 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.358919 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.359383 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.359489 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.361723 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:53.361729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:53.361732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:53.361760 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:53.364076 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.364620 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.364623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.364625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.364750 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.365015 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.365018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.365020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.365315 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:53.365489 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.365761 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.365905 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:21:53.369477 25727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:53.376073 25727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:53.377882 26085 heartbeater.cc:503] Master 127.22.121.62:44137 was elected leader, sending a full tablet report...
I20260811 06:21:53.409094 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.387730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.22.121.3:33989)
0811 06:21:53.401088 (+ 13358us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.401096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.401137 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:21:53.401232 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.401240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.401485 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.401672 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:53.402475 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.404215 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.406696 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.407950 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.408079 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.408294 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.408616 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.408771 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13198,"thread_start_us":13053,"threads_started":1}
I20260811 06:21:53.409180 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.387355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:33989 (local address 127.0.0.1:41682)
0811 06:21:53.400228 (+ 12873us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.400279 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.400454 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.401795 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.401801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.401829 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:53.402258 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.402272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.404329 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.404334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.406502 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.406517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.408144 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.408150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.408368 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.408425 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.408550 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.408715 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12617,"mutex_wait_us":1,"thread_start_us":13744,"threads_started":1}
I20260811 06:21:53.409691 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.387607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:45793 (local address 127.0.0.1:50870)
0811 06:21:53.401353 (+ 13746us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.401370 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.401476 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.403305 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.403309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.403335 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:53.403792 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.403807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.405435 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.405441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.407287 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.407304 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.408980 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.408986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.409171 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.409207 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.409271 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.409382 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13619,"thread_start_us":13951,"threads_started":1}
I20260811 06:21:53.410748 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.387812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.22.121.1:45793)
0811 06:21:53.402510 (+ 14698us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.402518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.402547 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:21:53.402637 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.402644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.402854 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.403030 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:53.403939 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.405312 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.407443 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.408733 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.410143 (+ 1410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.410352 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.410409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.410509 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14535,"thread_start_us":14394,"threads_started":1}
I20260811 06:21:53.411638 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.401912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:33001 (local address 127.0.0.1:57662)
0811 06:21:53.402723 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.402741 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.402846 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.404847 (+ 2001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.404851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.404877 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:53.405283 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.405297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.407085 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.407091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.408856 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.408870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.410864 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.410869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.411027 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.411058 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.411156 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.411296 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":24,"thread_start_us":1027,"threads_started":1}
I20260811 06:21:53.411711 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.402172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.22.121.2:33001)
0811 06:21:53.404322 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.404330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.404358 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:21:53.404442 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.404450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.404647 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.404822 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:53.405481 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.406949 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.409063 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.410626 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.410814 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.410985 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.411210 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.411392 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1996,"thread_start_us":1888,"threads_started":1}
I20260811 06:21:53.413457 26455 tablet_service.cc:1467] Processing CreateTablet for tablet e84f7fc80b2047aba760003209915293 (DEFAULT_TABLE table=test-table [id=dbdc8b9a1320494d80a981f58146ac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:53.413777 25961 tablet_service.cc:1467] Processing CreateTablet for tablet e84f7fc80b2047aba760003209915293 (DEFAULT_TABLE table=test-table [id=dbdc8b9a1320494d80a981f58146ac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:53.415011 25961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84f7fc80b2047aba760003209915293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:53.415026 26455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84f7fc80b2047aba760003209915293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:53.417136 26220 tablet_service.cc:1467] Processing CreateTablet for tablet e84f7fc80b2047aba760003209915293 (DEFAULT_TABLE table=test-table [id=dbdc8b9a1320494d80a981f58146ac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:53.418366 26220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84f7fc80b2047aba760003209915293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:53.425990 26580 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Bootstrap starting.
I20260811 06:21:53.428721 26581 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Bootstrap starting.
I20260811 06:21:53.429133 26580 tablet_bootstrap.cc:654] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:53.430457 26580 log.cc:824] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:53.432001 26581 tablet_bootstrap.cc:654] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:53.433281 26581 log.cc:824] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:53.433516 26580 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: No bootstrap required, opened a new log
I20260811 06:21:53.433724 26583 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Bootstrap starting.
I20260811 06:21:53.433851 26580 ts_tablet_manager.cc:1397] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260811 06:21:53.436580 26583 tablet_bootstrap.cc:654] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:53.437472 26583 log.cc:824] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:53.437742 26581 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: No bootstrap required, opened a new log
I20260811 06:21:53.438019 26581 ts_tablet_manager.cc:1397] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:21:53.443598 26583 tablet_bootstrap.cc:492] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: No bootstrap required, opened a new log
I20260811 06:21:53.443822 26583 ts_tablet_manager.cc:1397] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 06:21:53.444224 26580 raft_consensus.cc:348] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.444643 26580 raft_consensus.cc:374] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:53.444743 26580 raft_consensus.cc:729] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 684441c3f2d141e89cf35bdea0158b55, State: Initialized, Role: FOLLOWER
I20260811 06:21:53.445338 26580 consensus_queue.cc:260] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.446362 26555 heartbeater.cc:503] Master 127.22.121.62:44137 was elected leader, sending a full tablet report...
I20260811 06:21:53.448280 26580 ts_tablet_manager.cc:1428] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 06:21:53.448014 26581 raft_consensus.cc:348] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.448415 26581 raft_consensus.cc:374] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:53.448554 26581 raft_consensus.cc:729] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964420f356e14ca7a27388477481aabf, State: Initialized, Role: FOLLOWER
I20260811 06:21:53.449092 26581 consensus_queue.cc:260] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.450641 26581 ts_tablet_manager.cc:1428] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 06:21:53.452852 26583 raft_consensus.cc:348] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.453264 26583 raft_consensus.cc:374] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:53.453953 26583 raft_consensus.cc:729] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e36c185a9e3429cab9cdaaa74084dd6, State: Initialized, Role: FOLLOWER
I20260811 06:21:53.454509 26583 consensus_queue.cc:260] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.455473 26335 heartbeater.cc:503] Master 127.22.121.62:44137 was elected leader, sending a full tablet report...
I20260811 06:21:53.456053 26583 ts_tablet_manager.cc:1428] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
W20260811 06:21:53.567719 26557 tablet.cc:2369] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:21:53.577034 26336 tablet.cc:2369] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:21:53.601627 26086 tablet.cc:2369] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:53.735185 26589 raft_consensus.cc:482] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:53.735532 26589 raft_consensus.cc:504] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.737236 26589 leader_election.cc:288] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 684441c3f2d141e89cf35bdea0158b55 (127.22.121.3:33989), 0e36c185a9e3429cab9cdaaa74084dd6 (127.22.121.2:33001)
I20260811 06:21:53.744293 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.737690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:47651 (local address 127.22.121.3:33989)
0811 06:21:53.737864 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.737869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.738227 (+ 358us) server_negotiation.cc:407] Connection header received
0811 06:21:53.738634 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.738639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.738726 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.738833 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:53.739529 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.740679 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.742550 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.743365 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.743531 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.743606 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.743912 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.744049 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:21:53.744329 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.737470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:33989 (local address 127.22.121.1:47651)
0811 06:21:53.738108 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.738128 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.738280 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.738900 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.738905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.738933 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:53.739349 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.739362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.740810 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.740816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.742380 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.742395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.743569 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.743576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.743745 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.743780 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.743870 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.744005 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":300,"threads_started":1}
I20260811 06:21:53.745445 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.738204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:33001 (local address 127.22.121.1:43529)
0811 06:21:53.738861 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:53.738879 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:53.739078 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:53.739585 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:53.739590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:53.739614 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:53.739984 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.739996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.741038 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.741043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.743132 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:53.743148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.744732 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.744738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:53.744918 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.744941 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:21:53.745022 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:21:53.745136 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":320,"threads_started":1}
I20260811 06:21:53.745730 26489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84f7fc80b2047aba760003209915293" candidate_uuid: "964420f356e14ca7a27388477481aabf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "684441c3f2d141e89cf35bdea0158b55" is_pre_election: true
I20260811 06:21:53.746019 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.738322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:43529 (local address 127.22.121.2:33001)
0811 06:21:53.738490 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.738518 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.738955 (+ 437us) server_negotiation.cc:407] Connection header received
0811 06:21:53.739217 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.739221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.739338 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.739426 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:53.740145 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.740884 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.743331 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.744387 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.744531 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.744592 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.745076 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.745777 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 06:21:53.746256 26489 raft_consensus.cc:2454] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964420f356e14ca7a27388477481aabf in term 0.
I20260811 06:21:53.746901 25873 leader_election.cc:302] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 684441c3f2d141e89cf35bdea0158b55, 964420f356e14ca7a27388477481aabf; no voters:
I20260811 06:21:53.746855 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84f7fc80b2047aba760003209915293" candidate_uuid: "964420f356e14ca7a27388477481aabf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" is_pre_election: true
I20260811 06:21:53.747280 26589 raft_consensus.cc:2781] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:53.747349 26252 raft_consensus.cc:2454] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964420f356e14ca7a27388477481aabf in term 0.
I20260811 06:21:53.747433 26589 raft_consensus.cc:482] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:53.747535 26589 raft_consensus.cc:3037] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:53.749794 26589 raft_consensus.cc:504] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.750324 26589 leader_election.cc:288] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [CANDIDATE]: Term 1 election: Requested vote from peers 684441c3f2d141e89cf35bdea0158b55 (127.22.121.3:33989), 0e36c185a9e3429cab9cdaaa74084dd6 (127.22.121.2:33001)
I20260811 06:21:53.750809 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84f7fc80b2047aba760003209915293" candidate_uuid: "964420f356e14ca7a27388477481aabf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e36c185a9e3429cab9cdaaa74084dd6"
I20260811 06:21:53.750859 26489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84f7fc80b2047aba760003209915293" candidate_uuid: "964420f356e14ca7a27388477481aabf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "684441c3f2d141e89cf35bdea0158b55"
I20260811 06:21:53.751032 26489 raft_consensus.cc:3037] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:53.751032 26252 raft_consensus.cc:3037] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:53.754104 26489 raft_consensus.cc:2454] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964420f356e14ca7a27388477481aabf in term 1.
I20260811 06:21:53.754104 26252 raft_consensus.cc:2454] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964420f356e14ca7a27388477481aabf in term 1.
I20260811 06:21:53.754478 25873 leader_election.cc:302] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e36c185a9e3429cab9cdaaa74084dd6, 964420f356e14ca7a27388477481aabf; no voters:
I20260811 06:21:53.754818 26589 raft_consensus.cc:2781] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:53.755312 26589 raft_consensus.cc:686] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 1 LEADER]: Becoming Leader. State: Replica: 964420f356e14ca7a27388477481aabf, State: Running, Role: LEADER
I20260811 06:21:53.756048 26589 consensus_queue.cc:237] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } }
I20260811 06:21:53.760587 25726 catalog_manager.cc:5495] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf reported cstate change: term changed from 0 to 1, leader changed from <none> to 964420f356e14ca7a27388477481aabf (127.22.121.1). New cstate: current_term: 1 leader_uuid: "964420f356e14ca7a27388477481aabf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964420f356e14ca7a27388477481aabf" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 45793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 } health_report { overall_health: UNKNOWN } } }
I20260811 06:21:53.862622 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.857513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.22.121.1:45793)
0811 06:21:53.857712 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.857718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.857740 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:21:53.857835 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.857839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.857916 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.858051 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:53.858425 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.859150 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.860566 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.860969 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.861081 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.861555 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:53.862039 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:53.862168 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.862283 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.862365 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:53.870556 26252 raft_consensus.cc:1261] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 1 FOLLOWER]: Refusing update from remote peer 964420f356e14ca7a27388477481aabf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:53.870568 26489 raft_consensus.cc:1261] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 1 FOLLOWER]: Refusing update from remote peer 964420f356e14ca7a27388477481aabf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:53.871410 26589 consensus_queue.cc:1035] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e36c185a9e3429cab9cdaaa74084dd6" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:53.871726 26598 consensus_queue.cc:1035] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [LEADER]: Connected to new peer: Peer: permanent_uuid: "684441c3f2d141e89cf35bdea0158b55" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:53.884018 26607 mvcc.cc:205] Tried to move back new op lower bound from 7317214469604073472 to 7317214469148311552. Current Snapshot: MvccSnapshot[applied={T|T < 7317214469604073472}]
I20260811 06:21:53.896512 25726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:53.896809 25726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:53.900166 25726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2bc8314145c4d34adc54b886915abf7: Sending DeleteTablet for 3 replicas of tablet e84f7fc80b2047aba760003209915293
I20260811 06:21:53.901343 25961 tablet_service.cc:1514] Processing DeleteTablet for tablet e84f7fc80b2047aba760003209915293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:50870
I20260811 06:21:53.901386 26455 tablet_service.cc:1514] Processing DeleteTablet for tablet e84f7fc80b2047aba760003209915293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:41682
I20260811 06:21:53.901567 26220 tablet_service.cc:1514] Processing DeleteTablet for tablet e84f7fc80b2047aba760003209915293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:53 UTC) from {username='kudu', principal='kudu/127.22.121.62@KRBTEST.COM'} at 127.0.0.1:57662
I20260811 06:21:53.903314 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25843
I20260811 06:21:53.915920 26623 tablet_replica.cc:330] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: stopping tablet replica
I20260811 06:21:53.916862 26623 raft_consensus.cc:2229] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:53.917904 26623 raft_consensus.cc:2258] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:53.920780 26623 ts_tablet_manager.cc:1905] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:53.926074 26619 tablet_replica.cc:330] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: stopping tablet replica
I20260811 06:21:53.926553 26619 raft_consensus.cc:2229] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:53.927435 26619 raft_consensus.cc:2258] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:53.930075 26623 ts_tablet_manager.cc:1918] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:53.930708 26620 tablet_replica.cc:330] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: stopping tablet replica
I20260811 06:21:53.931087 26620 raft_consensus.cc:2229] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:53.931385 26619 ts_tablet_manager.cc:1905] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:53.931595 26620 raft_consensus.cc:2258] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:53.933068 26623 log.cc:1196] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/e84f7fc80b2047aba760003209915293
I20260811 06:21:53.933497 26620 ts_tablet_manager.cc:1905] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:53.937201 26623 ts_tablet_manager.cc:1939] T e84f7fc80b2047aba760003209915293 P 0e36c185a9e3429cab9cdaaa74084dd6: Deleting consensus metadata
I20260811 06:21:53.937871 26619 ts_tablet_manager.cc:1918] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:53.939045 25697 catalog_manager.cc:4841] TS 0e36c185a9e3429cab9cdaaa74084dd6 (127.22.121.2:33001): tablet e84f7fc80b2047aba760003209915293 (table test-table [id=dbdc8b9a1320494d80a981f58146ac57]) successfully deleted
I20260811 06:21:53.939597 26619 log.cc:1196] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/e84f7fc80b2047aba760003209915293
I20260811 06:21:53.940011 26620 ts_tablet_manager.cc:1918] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:53.940224 26620 log.cc:1196] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/e84f7fc80b2047aba760003209915293
I20260811 06:21:53.940239 26619 ts_tablet_manager.cc:1939] T e84f7fc80b2047aba760003209915293 P 964420f356e14ca7a27388477481aabf: Deleting consensus metadata
I20260811 06:21:53.940804 26620 ts_tablet_manager.cc:1939] T e84f7fc80b2047aba760003209915293 P 684441c3f2d141e89cf35bdea0158b55: Deleting consensus metadata
Aug 11 06:21:53 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.1@KRBTEST.COM
I20260811 06:21:53.941967 25700 catalog_manager.cc:4841] TS 964420f356e14ca7a27388477481aabf (127.22.121.1:45793): tablet e84f7fc80b2047aba760003209915293 (table test-table [id=dbdc8b9a1320494d80a981f58146ac57]) successfully deleted
I20260811 06:21:53.942401 25697 catalog_manager.cc:4841] TS 684441c3f2d141e89cf35bdea0158b55 (127.22.121.3:33989): tablet e84f7fc80b2047aba760003209915293 (table test-table [id=dbdc8b9a1320494d80a981f58146ac57]) successfully deleted
I20260811 06:21:53.948464 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:53.904497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.22.121.1:45793)
0811 06:21:53.904667 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:53.904673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:53.919888 (+ 15215us) server_negotiation.cc:407] Connection header received
0811 06:21:53.920321 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:53.920331 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:53.920420 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:53.920498 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:53.922368 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.923628 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.927667 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:53.928441 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:53.928537 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:53.942547 (+ 14010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:53.942562 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:53.942579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:53.942636 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:53.945584 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.946419 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.946425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.946428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.946588 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:53.946979 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:53.946983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:53.946985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:53.947330 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:53.947549 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:53.947930 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 06:21:53.948149 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:21:53.949148 26076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:54.046623 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25843
I20260811 06:21:54.070732 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26092
Aug 11 06:21:54 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.2@KRBTEST.COM
I20260811 06:21:54.084959 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:54.071520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.22.121.2:33001)
0811 06:21:54.071701 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:54.071706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:54.071726 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:21:54.071860 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:54.071864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:54.071927 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:54.072016 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:54.073306 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.074011 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.075235 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.075814 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.075937 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:54.080115 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:54.080128 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:54.080143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:54.080181 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:54.082629 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:54.083208 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:54.083215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:54.083217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:54.083469 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:54.083760 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:54.083763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:54.083765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:54.084117 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:54.084343 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:54.084547 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 06:21:54.084698 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:21:54.085567 26324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:54.171233 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26092
I20260811 06:21:54.194329 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26342
Aug 11 06:21:54 dist-test-slave-gjrz krb5kdc[25646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.3@KRBTEST.COM
I20260811 06:21:54.208279 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:54.195023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.22.121.3:33989)
0811 06:21:54.195176 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:54.195181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:54.195248 (+ 67us) server_negotiation.cc:407] Connection header received
0811 06:21:54.195601 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:54.195605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:54.195690 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:54.195773 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:54.197006 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.197725 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.198899 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.199309 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.199402 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:54.203485 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:54.203501 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:54.203520 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:54.203550 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:54.206125 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:54.206680 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:54.206687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:54.206689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:54.206828 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:54.207105 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:54.207108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:54.207110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:54.207470 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:54.207682 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:54.207880 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 06:21:54.208027 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:21:54.208920 26550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:54.295416 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26342
I20260811 06:21:54.315686 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 25670
I20260811 06:21:54.316746 25780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:54.422595 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 25670
2026-08-11T06:21:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4562 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T06:21:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:54Z Disabled control of system clock
I20260811 06:21:54.507756 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39019
--webserver_interface=127.22.121.62
--webserver_port=0
--builtin_ntp_servers=127.22.121.20:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:39019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:54.833029 26641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:54.833289 26641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:54.833411 26641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:54.840308 26641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:54.840410 26641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:54.840474 26641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:54.840547 26641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:54.840605 26641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:54.849400 26641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:39019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:39019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:54.850558 26641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:54.852286 26641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:54.863498 26655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:54.863567 26653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:54.863600 26651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:54.864487 26641 server_base.cc:1034] running on GCE node
I20260811 06:21:54.865168 26641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:54.866436 26641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:54.867643 26641 hybrid_clock.cc:648] HybridClock initialized: now 1786429314867593 us; error 41 us; skew 500 ppm
I20260811 06:21:54.867957 26641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:54.870018 26641 webserver.cc:460] Webserver started at http://127.22.121.62:37193/ using document root <none> and password file <none>
I20260811 06:21:54.870638 26641 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:54.870738 26641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:54.871025 26641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:54.873730 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "ade29d555a90455e9ee16594b3793348"
format_stamp: "Formatted at 2026-08-11 06:21:54 on dist-test-slave-gjrz"
server_key: "d2aa4cfa6a6fb91c8629f487767dfc24"
server_key_iv: "f5b92f557d5ae91abf949e8b5fe91422"
server_key_version: "encryptionkey@0"
I20260811 06:21:54.874476 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "ade29d555a90455e9ee16594b3793348"
format_stamp: "Formatted at 2026-08-11 06:21:54 on dist-test-slave-gjrz"
server_key: "d2aa4cfa6a6fb91c8629f487767dfc24"
server_key_iv: "f5b92f557d5ae91abf949e8b5fe91422"
server_key_version: "encryptionkey@0"
I20260811 06:21:54.879062 26641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:21:54.882227 26670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:54.883455 26641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:21:54.883611 26641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "ade29d555a90455e9ee16594b3793348"
format_stamp: "Formatted at 2026-08-11 06:21:54 on dist-test-slave-gjrz"
server_key: "d2aa4cfa6a6fb91c8629f487767dfc24"
server_key_iv: "f5b92f557d5ae91abf949e8b5fe91422"
server_key_version: "encryptionkey@0"
I20260811 06:21:54.883800 26641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:54.897958 26641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:54.898674 26641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:54.898913 26641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:54.915500 26641 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:39019
I20260811 06:21:54.915519 26756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:39019 every 8 connection(s)
I20260811 06:21:54.916683 26641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:54.918125 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26641
I20260811 06:21:54.918296 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:54.918860 23012 external_mini_cluster.cc:1428] Setting key f88066d040459336ac03dead5c57d60e
I20260811 06:21:54.920831 26757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:54.931121 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:54.920709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.22.121.62:39019)
0811 06:21:54.921635 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:54.921645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:54.921683 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:21:54.923221 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:54.923240 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:54.923682 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:54.924068 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:54.924612 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.925899 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.927182 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:54.927851 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:54.928010 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:54.929078 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:54.929085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:54.929098 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:54.929614 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:54.929686 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:54.929705 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:54.929802 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:54.930022 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 06:21:54.930287 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
I20260811 06:21:54.931366 26757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: Bootstrap starting.
I20260811 06:21:54.934326 26757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:54.935425 26757 log.cc:824] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:54.938197 26757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: No bootstrap required, opened a new log
I20260811 06:21:54.945683 26757 raft_consensus.cc:348] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } }
I20260811 06:21:54.946003 26757 raft_consensus.cc:374] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:54.946098 26757 raft_consensus.cc:729] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade29d555a90455e9ee16594b3793348, State: Initialized, Role: FOLLOWER
I20260811 06:21:54.946624 26757 consensus_queue.cc:260] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } }
I20260811 06:21:54.946841 26757 raft_consensus.cc:388] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:54.946964 26757 raft_consensus.cc:482] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:54.947129 26757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:54.949136 26757 raft_consensus.cc:504] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } }
I20260811 06:21:54.949592 26757 leader_election.cc:302] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ade29d555a90455e9ee16594b3793348; no voters:
I20260811 06:21:54.950122 26757 leader_election.cc:288] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:54.950248 26767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:54.950925 26767 raft_consensus.cc:686] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [term 1 LEADER]: Becoming Leader. State: Replica: ade29d555a90455e9ee16594b3793348, State: Running, Role: LEADER
I20260811 06:21:54.951448 26767 consensus_queue.cc:237] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } }
I20260811 06:21:54.952034 26757 sys_catalog.cc:564] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:54.954341 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ade29d555a90455e9ee16594b3793348. Latest consensus state: current_term: 1 leader_uuid: "ade29d555a90455e9ee16594b3793348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } } }
I20260811 06:21:54.954639 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:54.955856 26786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:54.956101 26771 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ade29d555a90455e9ee16594b3793348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade29d555a90455e9ee16594b3793348" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 39019 } } }
I20260811 06:21:54.956398 26771 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:54.959933 26786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:54.968614 26786 catalog_manager.cc:1347] Generated new cluster ID: 9c3662f1b32b40e0a3d2f27d4ba93902
I20260811 06:21:54.968711 26786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:54.988235 26786 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:54.989387 26786 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:54.998423 26786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: Generated new TSK 0
I20260811 06:21:54.999357 26786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:21:55.007707 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.1:0
--local_ip_for_outbound_sockets=127.22.121.1
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--builtin_ntp_servers=127.22.121.20:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:21:55.328392 26802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:55.328626 26802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:55.328714 26802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:55.335559 26802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:55.335641 26802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:55.335745 26802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.1
I20260811 06:21:55.344255 26802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.121.1
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.1
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:55.345342 26802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:55.346984 26802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:55.359107 26815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:55.359192 26818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:55.359287 26820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:55.360255 26802 server_base.cc:1034] running on GCE node
I20260811 06:21:55.360801 26802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:55.361654 26802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:55.362852 26802 hybrid_clock.cc:648] HybridClock initialized: now 1786429315362797 us; error 51 us; skew 500 ppm
I20260811 06:21:55.363137 26802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:55.365149 26802 webserver.cc:460] Webserver started at http://127.22.121.1:41219/ using document root <none> and password file <none>
I20260811 06:21:55.365759 26802 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:55.365859 26802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:55.366150 26802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:55.368875 26802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data/instance:
uuid: "0faddeaf69524ae28ad98abe0a198882"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "d1066a631eb57265a5b2b0dc3e0ff335"
server_key_iv: "715c7e9481e58e732971ecac81a3cf9d"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.369606 26802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance:
uuid: "0faddeaf69524ae28ad98abe0a198882"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "d1066a631eb57265a5b2b0dc3e0ff335"
server_key_iv: "715c7e9481e58e732971ecac81a3cf9d"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.374260 26802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:21:55.377389 26836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:55.378541 26802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:21:55.378677 26802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal
uuid: "0faddeaf69524ae28ad98abe0a198882"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "d1066a631eb57265a5b2b0dc3e0ff335"
server_key_iv: "715c7e9481e58e732971ecac81a3cf9d"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.378856 26802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:55.397004 26802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:55.397672 26802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:55.397938 26802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:55.398761 26802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:55.400075 26802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:55.400177 26802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:55.400272 26802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:55.400341 26802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:55.429777 26802 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.1:42171
I20260811 06:21:55.429842 27040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.1:42171 every 8 connection(s)
I20260811 06:21:55.430900 26802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/data/info.pb
I20260811 06:21:55.433672 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 26802
I20260811 06:21:55.433851 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal/instance
I20260811 06:21:55.434403 23012 external_mini_cluster.cc:1428] Setting key fb2c4049349f584f8f989af61425d91f
I20260811 06:21:55.437170 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.2:0
--local_ip_for_outbound_sockets=127.22.121.2
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--builtin_ntp_servers=127.22.121.20:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:55.443044 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:55.433189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.1:42989 (local address 127.22.121.62:39019)
0811 06:21:55.433620 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:55.433625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:55.434834 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 06:21:55.436812 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:55.436818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:55.436897 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:55.437018 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:55.438004 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.439003 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.440507 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.441029 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.441144 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:55.442316 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:55.442322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:55.442326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:55.442347 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:55.442392 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:55.442408 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:55.442495 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:55.442656 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:21:55.442744 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":223,"threads_started":1}
I20260811 06:21:55.443851 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:55.433744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39019 (local address 127.22.121.1:42989)
0811 06:21:55.434687 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:55.434733 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:55.436505 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:55.437169 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:55.437176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:55.437264 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:55.437840 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:55.437857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.439124 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.439128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:55.440363 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:55.440375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.441199 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.441203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:55.441396 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:55.442024 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:55.442026 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:55.442061 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:55.442135 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:55.442490 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:55.442496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:55.442500 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 06:21:55.442630 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:21:55.442889 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":289,"threads_started":1}
I20260811 06:21:55.445176 27048 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39019
I20260811 06:21:55.445446 27048 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:55.446110 27048 heartbeater.cc:511] Master 127.22.121.62:39019 requested a full tablet report, sending...
I20260811 06:21:55.448055 26699 ts_manager.cc:194] Registered new tserver with Master: 0faddeaf69524ae28ad98abe0a198882 (127.22.121.1:42171)
I20260811 06:21:55.449494 26699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.1:42989
W20260811 06:21:55.755962 27057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:55.756208 27057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:55.756306 27057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:55.763341 27057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:55.763424 27057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:55.763535 27057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.2
I20260811 06:21:55.772204 27057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.121.2
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.2
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:55.773265 27057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:55.774864 27057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:55.787014 27064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:55.787058 27066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:55.787092 27070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:55.788059 27057 server_base.cc:1034] running on GCE node
I20260811 06:21:55.788616 27057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:55.789448 27057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:55.790648 27057 hybrid_clock.cc:648] HybridClock initialized: now 1786429315790606 us; error 40 us; skew 500 ppm
I20260811 06:21:55.790930 27057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:55.792936 27057 webserver.cc:460] Webserver started at http://127.22.121.2:33881/ using document root <none> and password file <none>
I20260811 06:21:55.793569 27057 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:55.793676 27057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:55.793957 27057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:55.796655 27057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data/instance:
uuid: "e6a165dc70364d19b53d9bd229e3b3a2"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "17b6315a141c369b9aeef68cff12502e"
server_key_iv: "bf676fbc40caa4fbb4c6e81998286e79"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.797381 27057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance:
uuid: "e6a165dc70364d19b53d9bd229e3b3a2"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "17b6315a141c369b9aeef68cff12502e"
server_key_iv: "bf676fbc40caa4fbb4c6e81998286e79"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.802042 27057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:21:55.805181 27076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:55.806346 27057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:21:55.806488 27057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal
uuid: "e6a165dc70364d19b53d9bd229e3b3a2"
format_stamp: "Formatted at 2026-08-11 06:21:55 on dist-test-slave-gjrz"
server_key: "17b6315a141c369b9aeef68cff12502e"
server_key_iv: "bf676fbc40caa4fbb4c6e81998286e79"
server_key_version: "encryptionkey@0"
I20260811 06:21:55.806671 27057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:55.823982 27057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:55.824668 27057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:55.824927 27057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:55.825768 27057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:55.827044 27057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:55.827144 27057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:55.827245 27057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:55.827338 27057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 06:21:55.856876 27057 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.2:33047
I20260811 06:21:55.856935 27276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.2:33047 every 8 connection(s)
I20260811 06:21:55.858093 27057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/data/info.pb
I20260811 06:21:55.864558 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27057
I20260811 06:21:55.864745 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal/instance
I20260811 06:21:55.865249 23012 external_mini_cluster.cc:1428] Setting key 3d9c1b703e361cb1b0c4dca6d5387a04
I20260811 06:21:55.867888 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.121.3:0
--local_ip_for_outbound_sockets=127.22.121.3
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--builtin_ntp_servers=127.22.121.20:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:21:55.870164 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:55.860472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.2:46575 (local address 127.22.121.62:39019)
0811 06:21:55.860657 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:55.860662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:55.861982 (+ 1320us) server_negotiation.cc:407] Connection header received
0811 06:21:55.863686 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:55.863689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:55.863754 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:55.863836 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:55.865029 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.865768 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.867426 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.868034 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.868170 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:55.869456 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:55.869462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:55.869467 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:55.869488 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:55.869524 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:55.869540 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:55.869623 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:55.869779 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 06:21:55.869868 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:21:55.870985 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:55.860959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39019 (local address 127.22.121.2:46575)
0811 06:21:55.861821 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:55.861854 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:55.863471 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:55.864005 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:55.864012 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:55.864088 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:55.864871 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:55.864897 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.865925 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.865930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:55.867291 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:55.867306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:55.868287 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:55.868293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:55.868495 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:55.869158 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:55.869160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:55.869203 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:55.869276 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:55.869618 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:55.869624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:55.869629 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:21:55.869751 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:21:55.870049 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":272,"threads_started":1}
I20260811 06:21:55.872349 27278 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39019
I20260811 06:21:55.872615 27278 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:55.873323 27278 heartbeater.cc:511] Master 127.22.121.62:39019 requested a full tablet report, sending...
I20260811 06:21:55.874673 26699 ts_manager.cc:194] Registered new tserver with Master: e6a165dc70364d19b53d9bd229e3b3a2 (127.22.121.2:33047)
I20260811 06:21:55.875675 26699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.2:46575
W20260811 06:21:56.189005 27284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:56.189257 27284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:56.189363 27284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:56.196338 27284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:56.196419 27284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:56.196554 27284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.121.3
I20260811 06:21:56.205228 27284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.121.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.121.3
--webserver_port=0
--tserver_master_addrs=127.22.121.62:39019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.121.3
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:56.206351 27284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:56.208030 27284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:56.220472 27295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:56.220386 27294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:56.220435 27298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:56.221032 27284 server_base.cc:1034] running on GCE node
I20260811 06:21:56.221606 27284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:56.222466 27284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:56.223657 27284 hybrid_clock.cc:648] HybridClock initialized: now 1786429316223607 us; error 44 us; skew 500 ppm
I20260811 06:21:56.223937 27284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:21:56.226066 27284 webserver.cc:460] Webserver started at http://127.22.121.3:43771/ using document root <none> and password file <none>
I20260811 06:21:56.226675 27284 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:56.226773 27284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:56.227070 27284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:56.229743 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data/instance:
uuid: "0eed9fb19aa2433db0c59d658964bca4"
format_stamp: "Formatted at 2026-08-11 06:21:56 on dist-test-slave-gjrz"
server_key: "c0effec9dec012dde338bab66be32cd6"
server_key_iv: "982f0ae840fcce3d8650f0b32af43537"
server_key_version: "encryptionkey@0"
I20260811 06:21:56.230492 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance:
uuid: "0eed9fb19aa2433db0c59d658964bca4"
format_stamp: "Formatted at 2026-08-11 06:21:56 on dist-test-slave-gjrz"
server_key: "c0effec9dec012dde338bab66be32cd6"
server_key_iv: "982f0ae840fcce3d8650f0b32af43537"
server_key_version: "encryptionkey@0"
I20260811 06:21:56.235500 27284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:21:56.238708 27308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:56.239923 27284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:21:56.240082 27284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal
uuid: "0eed9fb19aa2433db0c59d658964bca4"
format_stamp: "Formatted at 2026-08-11 06:21:56 on dist-test-slave-gjrz"
server_key: "c0effec9dec012dde338bab66be32cd6"
server_key_iv: "982f0ae840fcce3d8650f0b32af43537"
server_key_version: "encryptionkey@0"
I20260811 06:21:56.240263 27284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:56.272957 27284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:56.273695 27284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:56.273953 27284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:56.274850 27284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:21:56.276281 27284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:21:56.276377 27284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:56.276479 27284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:21:56.276546 27284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:56.306394 27284 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.3:41561
I20260811 06:21:56.306473 27511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.3:41561 every 8 connection(s)
I20260811 06:21:56.307646 27284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/data/info.pb
I20260811 06:21:56.316306 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27284
I20260811 06:21:56.316457 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal/instance
I20260811 06:21:56.316876 23012 external_mini_cluster.cc:1428] Setting key eac5d4e3f4ea38f7c912909c41c906fc
I20260811 06:21:56.319069 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.309863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:38159 (local address 127.22.121.62:39019)
0811 06:21:56.310023 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.310051 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.311367 (+ 1316us) server_negotiation.cc:407] Connection header received
0811 06:21:56.313008 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.313012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.313083 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.313166 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:56.314106 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.314828 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.316390 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.316897 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.317015 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.318361 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:56.318366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:56.318369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:56.318386 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:56.318419 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:56.318436 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:56.318529 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.318733 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.318810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:21:56.320168 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.310343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:39019 (local address 127.22.121.3:38159)
0811 06:21:56.311192 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.311234 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.312796 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.313347 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.313354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.313432 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:21:56.313943 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.313958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.314959 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.314962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.316238 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.316254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.317127 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.317135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.317371 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.318084 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:21:56.318087 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:21:56.318122 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:21:56.318190 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:21:56.318539 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:21:56.318548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:21:56.318554 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.318727 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.319115 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":260,"threads_started":1}
I20260811 06:21:56.321274 27512 heartbeater.cc:347] Connected to a master server at 127.22.121.62:39019
I20260811 06:21:56.321527 27512 heartbeater.cc:464] Registering TS with master...
I20260811 06:21:56.322207 27512 heartbeater.cc:511] Master 127.22.121.62:39019 requested a full tablet report, sending...
I20260811 06:21:56.323473 26699 ts_manager.cc:194] Registered new tserver with Master: 0eed9fb19aa2433db0c59d658964bca4 (127.22.121.3:41561)
I20260811 06:21:56.324234 26699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.121.3:38159
I20260811 06:21:56.331751 23012 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:21:56.341007 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.335790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.22.121.62:39019)
0811 06:21:56.335965 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.335971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.336374 (+ 403us) server_negotiation.cc:407] Connection header received
0811 06:21:56.336509 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.336513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.336578 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.336649 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:56.337078 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.337728 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.339073 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.339494 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.339581 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.340450 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:56.340454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:56.340457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:56.340473 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:56.340501 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:56.340513 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:56.340577 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.340664 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.340728 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:21:56.344241 26699 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:21:56.350744 26699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:21:56.373430 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.361960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:33047 (local address 127.0.0.1:43764)
0811 06:21:56.363322 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.363364 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.363552 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.365001 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.365024 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.365064 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:56.365469 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.365483 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.367762 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.367769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.370479 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.370503 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.372500 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.372507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.372738 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.372792 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.372910 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.373099 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"threads_started":1}
I20260811 06:21:56.374769 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.362149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.22.121.2:33047)
0811 06:21:56.364073 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.364082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.364110 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:21:56.364212 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.364220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.364465 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.365696 (+ 1231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:56.365981 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.367589 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.370738 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.372267 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.373931 (+ 1664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.374178 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.374258 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.374421 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1781,"thread_start_us":1661,"threads_started":1}
I20260811 06:21:56.379078 27163 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 (DEFAULT_TABLE table=test-table [id=9513a956bec046bba3a5d4b007e6b1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:56.380389 27163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a56335ef4044e1c911354f7ec68bf52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:56.383998 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.362088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.3:41561 (local address 127.0.0.1:40948)
0811 06:21:56.373639 (+ 11551us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.373659 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.373758 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.377959 (+ 4201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.377963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.377992 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:56.378185 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.378197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.379877 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.379882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.381509 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.381532 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.383223 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.383230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.383438 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.383471 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.383575 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.383691 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11445,"mutex_wait_us":53,"thread_start_us":774,"threads_started":1}
I20260811 06:21:56.384243 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.362682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.22.121.3:41561)
0811 06:21:56.377400 (+ 14718us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.377409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.377438 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:21:56.377529 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.377536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.377744 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.378074 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:56.378299 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.379721 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.381661 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.383075 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.383207 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.383484 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.383618 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.383766 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14556,"thread_start_us":14408,"threads_started":1}
I20260811 06:21:56.387131 27404 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 (DEFAULT_TABLE table=test-table [id=9513a956bec046bba3a5d4b007e6b1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:56.388257 27404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a56335ef4044e1c911354f7ec68bf52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:56.390592 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.364071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:42171 (local address 127.0.0.1:37236)
0811 06:21:56.375494 (+ 11423us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.375512 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.375627 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.377826 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.377831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.377856 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:56.378502 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.378514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.380174 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.380178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.382991 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.383003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.385174 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.385180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.390035 (+ 4855us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.390068 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.390153 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.390268 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11266,"mutex_wait_us":2,"thread_start_us":13191,"threads_started":1}
I20260811 06:21:56.391429 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.364373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.22.121.1:42171)
0811 06:21:56.376994 (+ 12621us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.377002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.377031 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:21:56.377120 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.377127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.377351 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.377554 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:56.378702 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.380054 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.383279 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.384695 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.384807 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.385032 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.390922 (+ 5890us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.391085 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12458,"thread_start_us":12337,"threads_started":1}
I20260811 06:21:56.392426 27546 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Bootstrap starting.
I20260811 06:21:56.394531 26926 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 (DEFAULT_TABLE table=test-table [id=9513a956bec046bba3a5d4b007e6b1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:21:56.395433 27546 tablet_bootstrap.cc:654] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:56.395818 26926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a56335ef4044e1c911354f7ec68bf52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:56.396834 27546 log.cc:824] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:56.399132 27546 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: No bootstrap required, opened a new log
I20260811 06:21:56.399425 27546 ts_tablet_manager.cc:1397] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 06:21:56.403168 27552 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Bootstrap starting.
I20260811 06:21:56.406453 27552 tablet_bootstrap.cc:654] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:56.406551 27546 raft_consensus.cc:348] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.406886 27546 raft_consensus.cc:374] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:56.406978 27546 raft_consensus.cc:729] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a165dc70364d19b53d9bd229e3b3a2, State: Initialized, Role: FOLLOWER
I20260811 06:21:56.407503 27546 consensus_queue.cc:260] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.407626 27552 log.cc:824] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:56.408490 27278 heartbeater.cc:503] Master 127.22.121.62:39019 was elected leader, sending a full tablet report...
I20260811 06:21:56.409161 27546 ts_tablet_manager.cc:1428] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:21:56.409377 27560 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Bootstrap starting.
I20260811 06:21:56.411868 27552 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: No bootstrap required, opened a new log
I20260811 06:21:56.412262 27552 ts_tablet_manager.cc:1397] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:21:56.412926 27560 tablet_bootstrap.cc:654] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:56.414054 27560 log.cc:824] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:56.419418 27552 raft_consensus.cc:348] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.419713 27552 raft_consensus.cc:374] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:56.419816 27552 raft_consensus.cc:729] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eed9fb19aa2433db0c59d658964bca4, State: Initialized, Role: FOLLOWER
I20260811 06:21:56.420286 27552 consensus_queue.cc:260] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.421252 27512 heartbeater.cc:503] Master 127.22.121.62:39019 was elected leader, sending a full tablet report...
I20260811 06:21:56.421824 27560 tablet_bootstrap.cc:492] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: No bootstrap required, opened a new log
I20260811 06:21:56.421872 27552 ts_tablet_manager.cc:1428] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 06:21:56.422096 27560 ts_tablet_manager.cc:1397] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260811 06:21:56.428779 27560 raft_consensus.cc:348] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.429056 27560 raft_consensus.cc:374] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:56.429164 27560 raft_consensus.cc:729] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faddeaf69524ae28ad98abe0a198882, State: Initialized, Role: FOLLOWER
I20260811 06:21:56.429749 27560 consensus_queue.cc:260] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.430984 27048 heartbeater.cc:503] Master 127.22.121.62:39019 was elected leader, sending a full tablet report...
I20260811 06:21:56.431504 27560 ts_tablet_manager.cc:1428] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Time spent starting tablet: real 0.009s user 0.009s sys 0.002s
W20260811 06:21:56.432921 27049 tablet.cc:2369] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:56.540942 27565 raft_consensus.cc:482] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:21:56.541239 27565 raft_consensus.cc:504] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.542961 27565 leader_election.cc:288] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6a165dc70364d19b53d9bd229e3b3a2 (127.22.121.2:33047), 0faddeaf69524ae28ad98abe0a198882 (127.22.121.1:42171)
I20260811 06:21:56.548481 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.543147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.2:33047 (local address 127.22.121.3:51323)
0811 06:21:56.543351 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.543371 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.543498 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.543949 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.543955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.543985 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:56.544209 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.544226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.545235 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.545240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.546913 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.546928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.547970 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.547974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.548092 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.548114 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.548180 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.548272 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":1}
I20260811 06:21:56.548586 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.543416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:51323 (local address 127.22.121.2:33047)
0811 06:21:56.543602 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.543608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.543628 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:21:56.543700 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.543705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.543780 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.543899 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:56.544380 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.545094 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.547067 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.547837 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.548001 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.548069 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.548225 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.548347 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:21:56.549712 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.543556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.121.3:60195 (local address 127.22.121.1:42171)
0811 06:21:56.543716 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.543720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.544482 (+ 762us) server_negotiation.cc:407] Connection header received
0811 06:21:56.544624 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.544628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.544694 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.544766 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:21:56.545285 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.545935 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.547868 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.548976 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.549121 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.549168 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.549445 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.549524 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:21:56.549813 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.543547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.1:42171 (local address 127.22.121.3:60195)
0811 06:21:56.544380 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 06:21:56.544396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:21:56.544505 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:21:56.544794 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:21:56.544798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:21:56.544820 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:21:56.545109 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.545117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.546075 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.546081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.547691 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:21:56.547702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.549132 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.549138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:21:56.549311 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.549341 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:21:56.549433 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:21:56.549545 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":432,"threads_started":1}
I20260811 06:21:56.549602 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56335ef4044e1c911354f7ec68bf52" candidate_uuid: "0eed9fb19aa2433db0c59d658964bca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" is_pre_election: true
I20260811 06:21:56.550163 27199 raft_consensus.cc:2454] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eed9fb19aa2433db0c59d658964bca4 in term 0.
I20260811 06:21:56.550783 27311 leader_election.cc:302] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eed9fb19aa2433db0c59d658964bca4, e6a165dc70364d19b53d9bd229e3b3a2; no voters:
I20260811 06:21:56.550793 26965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56335ef4044e1c911354f7ec68bf52" candidate_uuid: "0eed9fb19aa2433db0c59d658964bca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faddeaf69524ae28ad98abe0a198882" is_pre_election: true
I20260811 06:21:56.551118 27565 raft_consensus.cc:2781] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:21:56.551254 27565 raft_consensus.cc:482] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:21:56.551322 26965 raft_consensus.cc:2454] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eed9fb19aa2433db0c59d658964bca4 in term 0.
I20260811 06:21:56.551397 27565 raft_consensus.cc:3037] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:56.553637 27565 raft_consensus.cc:504] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.554137 27565 leader_election.cc:288] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [CANDIDATE]: Term 1 election: Requested vote from peers e6a165dc70364d19b53d9bd229e3b3a2 (127.22.121.2:33047), 0faddeaf69524ae28ad98abe0a198882 (127.22.121.1:42171)
I20260811 06:21:56.554533 26965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56335ef4044e1c911354f7ec68bf52" candidate_uuid: "0eed9fb19aa2433db0c59d658964bca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faddeaf69524ae28ad98abe0a198882"
I20260811 06:21:56.554514 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56335ef4044e1c911354f7ec68bf52" candidate_uuid: "0eed9fb19aa2433db0c59d658964bca4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a165dc70364d19b53d9bd229e3b3a2"
I20260811 06:21:56.554713 27199 raft_consensus.cc:3037] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:56.554713 26965 raft_consensus.cc:3037] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:56.557821 27199 raft_consensus.cc:2454] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eed9fb19aa2433db0c59d658964bca4 in term 1.
I20260811 06:21:56.557816 26965 raft_consensus.cc:2454] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eed9fb19aa2433db0c59d658964bca4 in term 1.
I20260811 06:21:56.558192 27312 leader_election.cc:302] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eed9fb19aa2433db0c59d658964bca4, 0faddeaf69524ae28ad98abe0a198882; no voters:
I20260811 06:21:56.558466 27565 raft_consensus.cc:2781] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:56.558854 27565 raft_consensus.cc:686] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 1 LEADER]: Becoming Leader. State: Replica: 0eed9fb19aa2433db0c59d658964bca4, State: Running, Role: LEADER
W20260811 06:21:56.559161 27513 tablet.cc:2369] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:56.559473 27565 consensus_queue.cc:237] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } }
I20260811 06:21:56.563635 26699 catalog_manager.cc:5495] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0eed9fb19aa2433db0c59d658964bca4 (127.22.121.3). New cstate: current_term: 1 leader_uuid: "0eed9fb19aa2433db0c59d658964bca4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0eed9fb19aa2433db0c59d658964bca4" member_type: VOTER last_known_addr { host: "127.22.121.3" port: 41561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 } health_report { overall_health: UNKNOWN } } }
W20260811 06:21:56.610105 27279 tablet.cc:2369] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:21:56.618021 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.612942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.22.121.3:41561)
0811 06:21:56.613127 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.613133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.613234 (+ 101us) server_negotiation.cc:407] Connection header received
0811 06:21:56.613439 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.613442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.613516 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.613656 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:21:56.613999 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.614724 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.616091 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.616494 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.616608 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.617043 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:21:56.617491 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:21:56.617633 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.617722 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.617797 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:21:56.625449 27199 raft_consensus.cc:1261] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 1 FOLLOWER]: Refusing update from remote peer 0eed9fb19aa2433db0c59d658964bca4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:56.625488 26965 raft_consensus.cc:1261] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 1 FOLLOWER]: Refusing update from remote peer 0eed9fb19aa2433db0c59d658964bca4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:21:56.626299 27570 consensus_queue.cc:1035] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a165dc70364d19b53d9bd229e3b3a2" member_type: VOTER last_known_addr { host: "127.22.121.2" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:56.626596 27565 consensus_queue.cc:1035] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faddeaf69524ae28ad98abe0a198882" member_type: VOTER last_known_addr { host: "127.22.121.1" port: 42171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:21:56.653133 26699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:56.653429 26699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:21:56.656915 26699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ade29d555a90455e9ee16594b3793348: Sending DeleteTablet for 3 replicas of tablet 4a56335ef4044e1c911354f7ec68bf52
I20260811 06:21:56.657866 27163 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:56 UTC) from {username='slave'} at 127.0.0.1:43764
I20260811 06:21:56.657971 27404 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:56 UTC) from {username='slave'} at 127.0.0.1:40948
I20260811 06:21:56.658325 26926 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56335ef4044e1c911354f7ec68bf52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:21:56 UTC) from {username='slave'} at 127.0.0.1:37236
I20260811 06:21:56.658701 27598 tablet_replica.cc:330] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: stopping tablet replica
I20260811 06:21:56.659109 27598 raft_consensus.cc:2229] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:56.659190 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26802
I20260811 06:21:56.659641 27598 raft_consensus.cc:2258] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:56.669406 27598 ts_tablet_manager.cc:1905] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:56.671769 27604 tablet_replica.cc:330] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: stopping tablet replica
I20260811 06:21:56.672190 27604 raft_consensus.cc:2229] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:21:56.672691 27604 raft_consensus.cc:2258] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:56.673405 27600 tablet_replica.cc:330] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: stopping tablet replica
I20260811 06:21:56.673753 27600 raft_consensus.cc:2229] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:21:56.674422 27600 raft_consensus.cc:2258] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:21:56.676303 27604 ts_tablet_manager.cc:1905] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:56.676898 27598 ts_tablet_manager.cc:1918] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:56.677054 27598 log.cc:1196] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-1/wal/wals/4a56335ef4044e1c911354f7ec68bf52
I20260811 06:21:56.677560 27598 ts_tablet_manager.cc:1939] T 4a56335ef4044e1c911354f7ec68bf52 P e6a165dc70364d19b53d9bd229e3b3a2: Deleting consensus metadata
I20260811 06:21:56.678038 27600 ts_tablet_manager.cc:1905] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:21:56.679137 26671 catalog_manager.cc:4841] TS e6a165dc70364d19b53d9bd229e3b3a2 (127.22.121.2:33047): tablet 4a56335ef4044e1c911354f7ec68bf52 (table test-table [id=9513a956bec046bba3a5d4b007e6b1b5]) successfully deleted
I20260811 06:21:56.682375 27604 ts_tablet_manager.cc:1918] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:56.682628 27604 log.cc:1196] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-0/wal/wals/4a56335ef4044e1c911354f7ec68bf52
I20260811 06:21:56.683319 27604 ts_tablet_manager.cc:1939] T 4a56335ef4044e1c911354f7ec68bf52 P 0faddeaf69524ae28ad98abe0a198882: Deleting consensus metadata
I20260811 06:21:56.683770 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.661539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.22.121.1:42171)
0811 06:21:56.661715 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.661721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.672041 (+ 10320us) server_negotiation.cc:407] Connection header received
0811 06:21:56.672316 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.672321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.672495 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.672575 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:56.673435 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.674533 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.679384 (+ 4851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.680073 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.680173 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.681972 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:56.681982 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:56.681996 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:56.682742 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:56.682803 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:56.682827 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:56.682911 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.683230 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.683410 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:21:56.684814 27034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:56.686892 27600 ts_tablet_manager.cc:1918] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:21:56.687116 27600 log.cc:1196] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Deleting WAL directory at /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429000386903-23012-0/minicluster-data/ts-2/wal/wals/4a56335ef4044e1c911354f7ec68bf52
I20260811 06:21:56.687879 27600 ts_tablet_manager.cc:1939] T 4a56335ef4044e1c911354f7ec68bf52 P 0eed9fb19aa2433db0c59d658964bca4: Deleting consensus metadata
I20260811 06:21:56.689884 26674 catalog_manager.cc:4841] TS 0eed9fb19aa2433db0c59d658964bca4 (127.22.121.3:41561): tablet 4a56335ef4044e1c911354f7ec68bf52 (table test-table [id=9513a956bec046bba3a5d4b007e6b1b5]) successfully deleted
I20260811 06:21:56.814815 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26802
I20260811 06:21:56.814939 26672 catalog_manager.cc:4841] TS 0faddeaf69524ae28ad98abe0a198882 (127.22.121.1:42171): tablet 4a56335ef4044e1c911354f7ec68bf52 (table test-table [id=9513a956bec046bba3a5d4b007e6b1b5]) successfully deleted
I20260811 06:21:56.837291 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27057
I20260811 06:21:56.844321 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.838130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.22.121.2:33047)
0811 06:21:56.838313 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.838319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.838339 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:21:56.838430 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.838435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.838533 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.838618 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:56.839035 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.839839 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.841069 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.841581 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.841693 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.842693 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:56.842702 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:56.842736 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:56.843543 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:56.843592 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:56.843613 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:56.843710 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.843875 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.843980 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:21:56.844993 27269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:56.940156 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27057
I20260811 06:21:56.967206 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27284
I20260811 06:21:56.974920 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:56.968233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.22.121.3:41561)
0811 06:21:56.968442 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:56.968447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:56.968462 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:21:56.968518 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:56.968522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:56.968617 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:56.968716 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:56.969163 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.970033 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.971461 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:56.972144 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:56.972294 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:56.973304 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:56.973317 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:56.973336 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:21:56.974166 (+ 830us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:56.974234 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:21:56.974262 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:56.974382 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:56.974433 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 06:21:56.974535 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:21:56.975673 27504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:57.092675 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27284
I20260811 06:21:57.117403 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 26641
I20260811 06:21:57.118453 26744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:57.220548 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 26641
2026-08-11T06:21:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2796 ms)
[----------] 6 tests from EncryptionPolicyTest (21689 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:21:57 dist-test-slave-gjrz krb5kdc[27624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:21:57 dist-test-slave-gjrz krb5kdc[27624](info): set up 2 sockets
Aug 11 06:21:57 dist-test-slave-gjrz krb5kdc[27624](info): commencing operation
krb5kdc: starting...
W20260811 06:21:58.468945 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:21:58 dist-test-slave-gjrz krb5kdc[27624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:21:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:21:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:21:58.704160 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:21:59.020902 27656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:21:59.021174 27656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:21:59.021311 27656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:21:59.028223 27656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:21:59.028338 27656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:21:59.028411 27656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:21:59.028494 27656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:21:59.028553 27656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:21:59.037757 27656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:21:59.038830 27656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:21:59.040083 27656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:21:59.051164 27664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:59.051231 27662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:21:59.051208 27666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:21:59.052340 27656 server_base.cc:1034] running on GCE node
I20260811 06:21:59.053043 27656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:21:59.054256 27656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:21:59.055446 27656 hybrid_clock.cc:648] HybridClock initialized: now 1786429319055408 us; error 31 us; skew 500 ppm
I20260811 06:21:59.055737 27656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:21:59 dist-test-slave-gjrz krb5kdc[27624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:21:59.058590 27656 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:21:59.059985 27656 webserver.cc:460] Webserver started at http://127.0.0.1:36531/ using document root <none> and password file <none>
I20260811 06:21:59.060598 27656 fs_manager.cc:359] Metadata directory not provided
I20260811 06:21:59.060717 27656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:21:59.060992 27656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:21:59.063824 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "c5efe0a620fc46ac90654802cc9de531"
format_stamp: "Formatted at 2026-08-11 06:21:59 on dist-test-slave-gjrz"
server_key: "9463b6c1984345f97214ba8583235ce2"
server_key_iv: "5d731c47755c357c0bec207588a32aeb"
server_key_version: "encryptionkey@0"
I20260811 06:21:59.064546 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "c5efe0a620fc46ac90654802cc9de531"
format_stamp: "Formatted at 2026-08-11 06:21:59 on dist-test-slave-gjrz"
server_key: "9463b6c1984345f97214ba8583235ce2"
server_key_iv: "5d731c47755c357c0bec207588a32aeb"
server_key_version: "encryptionkey@0"
I20260811 06:21:59.069203 27656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:21:59.072342 27675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:21:59.073536 27656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:21:59.073745 27656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "c5efe0a620fc46ac90654802cc9de531"
format_stamp: "Formatted at 2026-08-11 06:21:59 on dist-test-slave-gjrz"
server_key: "9463b6c1984345f97214ba8583235ce2"
server_key_iv: "5d731c47755c357c0bec207588a32aeb"
server_key_version: "encryptionkey@0"
I20260811 06:21:59.073904 27656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:21:59.086670 27656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:21:59.090385 27656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:21:59.090642 27656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:21:59.107877 27656 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39071
I20260811 06:21:59.107920 27760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39071 every 8 connection(s)
I20260811 06:21:59.109166 27656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:21:59.112587 27761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:21:59.113781 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27656
I20260811 06:21:59.113950 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:21:59.114569 23012 external_mini_cluster.cc:1428] Setting key be499cebb2696fd3583e90afa90976c8
I20260811 06:21:59.125631 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531: Bootstrap starting.
Aug 11 06:21:59 dist-test-slave-gjrz krb5kdc[27624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:21:59.129151 27761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531: Neither blocks nor log segments found. Creating new log.
I20260811 06:21:59.130357 27761 log.cc:824] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531: Log is configured to *not* fsync() on all Append() calls
I20260811 06:21:59.133474 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531: No bootstrap required, opened a new log
I20260811 06:21:59.137969 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:59.116343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39487 (local address 127.0.0.1:39071)
0811 06:21:59.117160 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:59.117173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:59.117217 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:21:59.118754 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:59.118779 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:59.119145 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:59.119501 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:59.121237 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:59.122601 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:59.124436 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:59.125087 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:59.125242 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:59.130092 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:59.130100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:59.130113 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:59.130147 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:59.133618 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:59.134368 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:59.134376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:59.134383 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:59.134596 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:59.134993 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:59.134998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:59.135001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:59.135727 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:59.136051 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:59.136372 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:21:59.136712 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":354,"threads_started":1}
I20260811 06:21:59.141762 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:21:59.142664 27761 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260811 06:21:59.142967 27761 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:21:59.143066 27761 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5efe0a620fc46ac90654802cc9de531, State: Initialized, Role: FOLLOWER
I20260811 06:21:59.143638 27761 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260811 06:21:59.143857 27761 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:21:59.143983 27761 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:21:59.144125 27761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:21:59.146605 27761 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260811 06:21:59.147128 27761 leader_election.cc:302] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5efe0a620fc46ac90654802cc9de531; no voters:
I20260811 06:21:59.147729 27761 leader_election.cc:288] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:21:59.147822 27784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:21:59.149262 27784 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [term 1 LEADER]: Becoming Leader. State: Replica: c5efe0a620fc46ac90654802cc9de531, State: Running, Role: LEADER
I20260811 06:21:59.149909 27761 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:21:59.149818 27784 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260811 06:21:59.154212 27785 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5efe0a620fc46ac90654802cc9de531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260811 06:21:59.154587 27785 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:59.155107 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5efe0a620fc46ac90654802cc9de531. Latest consensus state: current_term: 1 leader_uuid: "c5efe0a620fc46ac90654802cc9de531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5efe0a620fc46ac90654802cc9de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260811 06:21:59.155428 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531 [sys.catalog]: This master's current role is: LEADER
I20260811 06:21:59.162521 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:21:59.146358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46703 (local address 127.0.0.1:39071)
0811 06:21:59.146483 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 06:21:59.146487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:21:59.147086 (+ 599us) server_negotiation.cc:407] Connection header received
0811 06:21:59.147217 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:21:59.147221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:21:59.147308 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:21:59.147384 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:21:59.149026 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:59.150069 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:59.151475 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:21:59.151984 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:21:59.152103 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:21:59.154850 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:21:59.154856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:21:59.154861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:21:59.154894 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:21:59.158457 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:59.159446 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:59.159453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:59.159457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:59.159634 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:21:59.160190 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:21:59.160195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:21:59.160199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:21:59.160578 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:21:59.160842 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 06:21:59.161964 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0811 06:21:59.162145 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 06:21:59.168325 27799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:21:59.172643 27799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:21:59.180864 27799 catalog_manager.cc:1347] Generated new cluster ID: 672607f844894785bb220f18c924065c
I20260811 06:21:59.180969 27799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:21:59.191433 27799 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:21:59.192561 27799 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:21:59.198426 27799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5efe0a620fc46ac90654802cc9de531: Generated new TSK 0
I20260811 06:21:59.199131 27799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:21:59.213464 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27656
I20260811 06:21:59.214804 27751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:21:59.302153 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27656
W20260811 06:21:59.326051 27618 connection.cc:535] client connection to 127.0.0.1:39071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39071): Connection reset by peer (error 104)
2026-08-11T06:21:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:21:59 dist-test-slave-gjrz krb5kdc[27829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:21:59 dist-test-slave-gjrz krb5kdc[27829](info): set up 2 sockets
Aug 11 06:21:59 dist-test-slave-gjrz krb5kdc[27829](info): commencing operation
krb5kdc: starting...
W20260811 06:22:00.537236 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:00 dist-test-slave-gjrz krb5kdc[27829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:00.778975 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:01.098387 27851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:01.098649 27851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:01.098768 27851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:01.105827 27851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:01.105921 27851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:01.106000 27851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:01.106081 27851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:01.106138 27851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:01.115557 27851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:01.116679 27851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:01.117882 27851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:01.129259 27866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:01.129300 27867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:01.129325 27870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:01.130479 27851 server_base.cc:1034] running on GCE node
I20260811 06:22:01.131172 27851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:01.132475 27851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:01.133682 27851 hybrid_clock.cc:648] HybridClock initialized: now 1786429321133639 us; error 37 us; skew 500 ppm
I20260811 06:22:01.133984 27851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:01 dist-test-slave-gjrz krb5kdc[27829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:01.140501 27851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:01.141804 27851 webserver.cc:460] Webserver started at http://127.0.0.1:42321/ using document root <none> and password file <none>
I20260811 06:22:01.142391 27851 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:01.142488 27851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:01.142745 27851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:01.145578 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "62b6ab4c5dfa47bbab2893654501ebc8"
format_stamp: "Formatted at 2026-08-11 06:22:01 on dist-test-slave-gjrz"
server_key: "654ff0fa9f8a5b24761c0821d6daa415"
server_key_iv: "fdb470a89bcd157888c6f12fc87ed3a7"
server_key_version: "encryptionkey@0"
I20260811 06:22:01.146263 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "62b6ab4c5dfa47bbab2893654501ebc8"
format_stamp: "Formatted at 2026-08-11 06:22:01 on dist-test-slave-gjrz"
server_key: "654ff0fa9f8a5b24761c0821d6daa415"
server_key_iv: "fdb470a89bcd157888c6f12fc87ed3a7"
server_key_version: "encryptionkey@0"
I20260811 06:22:01.151064 27851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 06:22:01.154356 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:01.155586 27851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:22:01.155735 27851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "62b6ab4c5dfa47bbab2893654501ebc8"
format_stamp: "Formatted at 2026-08-11 06:22:01 on dist-test-slave-gjrz"
server_key: "654ff0fa9f8a5b24761c0821d6daa415"
server_key_iv: "fdb470a89bcd157888c6f12fc87ed3a7"
server_key_version: "encryptionkey@0"
I20260811 06:22:01.155894 27851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:01.170222 27851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:01.173661 27851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:01.173900 27851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:01.190689 27851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33055
I20260811 06:22:01.190728 27984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33055 every 8 connection(s)
I20260811 06:22:01.191910 27851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:01.195227 27987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:01.199409 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 27851
I20260811 06:22:01.199591 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:01.200145 23012 external_mini_cluster.cc:1428] Setting key 4f65dad0b5a0710e5c36220bfcf08e3f
I20260811 06:22:01.205940 27987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8: Bootstrap starting.
I20260811 06:22:01.209281 27987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:01.210435 27987 log.cc:824] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:01.213176 27992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:01.201788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54869 (local address 127.0.0.1:33055)
0811 06:22:01.202525 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:01.202538 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:01.202591 (+ 53us) server_negotiation.cc:407] Connection header received
0811 06:22:01.204571 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:01.204594 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:01.204929 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:01.205228 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:01.205775 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:01.207076 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:01.208876 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:01.209564 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:01.209703 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:01.210867 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:01.210876 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:01.210893 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:01.211646 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:01.211736 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:01.211766 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:01.211882 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:01.212126 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 06:22:01.212281 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":335,"threads_started":1}
I20260811 06:22:01.213269 27987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8: No bootstrap required, opened a new log
I20260811 06:22:01.216756 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:01.222165 27987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260811 06:22:01.222599 27987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:01.222692 27987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b6ab4c5dfa47bbab2893654501ebc8, State: Initialized, Role: FOLLOWER
I20260811 06:22:01.223344 27987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260811 06:22:01.223635 27987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:01.223767 27987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:01.223950 27987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:01.226414 27987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260811 06:22:01.226852 27987 leader_election.cc:302] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62b6ab4c5dfa47bbab2893654501ebc8; no voters:
I20260811 06:22:01.227455 27987 leader_election.cc:288] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:01.227557 28004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:01.227869 27992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:01.221548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40071 (local address 127.0.0.1:33055)
0811 06:22:01.221699 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:01.221704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:01.222168 (+ 464us) server_negotiation.cc:407] Connection header received
0811 06:22:01.222294 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:01.222299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:01.222368 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:01.222488 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:01.222968 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:01.224066 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:01.225336 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:01.226004 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:01.226117 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:01.227084 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:01.227096 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:01.227099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:01.227115 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:01.227149 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:01.227164 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:01.227304 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:01.227462 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 06:22:01.227512 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:22:01.229684 27987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:01.231513 28004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [term 1 LEADER]: Becoming Leader. State: Replica: 62b6ab4c5dfa47bbab2893654501ebc8, State: Running, Role: LEADER
I20260811 06:22:01.232087 28004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260811 06:22:01.234912 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } } }
I20260811 06:22:01.235363 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:01.235869 28006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b6ab4c5dfa47bbab2893654501ebc8. Latest consensus state: current_term: 1 leader_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b6ab4c5dfa47bbab2893654501ebc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } } }
I20260811 06:22:01.236236 28006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:01.240754 28013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:01.245651 28013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:01.254106 28013 catalog_manager.cc:1347] Generated new cluster ID: c7e7d7feee154346934bc52c6a947be8
I20260811 06:22:01.254194 28013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:01.264019 28013 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:01.265142 28013 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:01.283715 28013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62b6ab4c5dfa47bbab2893654501ebc8: Generated new TSK 0
I20260811 06:22:01.284426 28013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:01.296893 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27851
W20260811 06:22:01.298717 27977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54869
W20260811 06:22:01.299407 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 27851: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:01.299593 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 27851
W20260811 06:22:01.322266 27821 connection.cc:535] client connection to 127.0.0.1:33055 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33055): Connection reset by peer (error 104)
2026-08-11T06:22:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1996 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:01 dist-test-slave-gjrz krb5kdc[28037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:01 dist-test-slave-gjrz krb5kdc[28037](info): set up 2 sockets
Aug 11 06:22:01 dist-test-slave-gjrz krb5kdc[28037](info): commencing operation
krb5kdc: starting...
W20260811 06:22:02.531445 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:02 dist-test-slave-gjrz krb5kdc[28037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:02.767598 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:03.082337 28057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:03.082607 28057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:03.082738 28057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:03.089663 28057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:03.089773 28057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:03.089848 28057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:03.089927 28057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:03.089991 28057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:03.099236 28057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:03.100353 28057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:03.101548 28057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:03.112872 28070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:03.112934 28065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:03.112973 28066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:03.113557 28057 server_base.cc:1034] running on GCE node
I20260811 06:22:03.114253 28057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:03.115535 28057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:03.116730 28057 hybrid_clock.cc:648] HybridClock initialized: now 1786429323116690 us; error 29 us; skew 500 ppm
I20260811 06:22:03.117038 28057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:03 dist-test-slave-gjrz krb5kdc[28037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:03.119913 28057 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:03.121214 28057 webserver.cc:460] Webserver started at http://127.0.0.1:33425/ using document root <none> and password file <none>
I20260811 06:22:03.121809 28057 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:03.121950 28057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:03.122248 28057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:03.125092 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "ee05703f7fd24ff388a962da7cfe6e3f"
format_stamp: "Formatted at 2026-08-11 06:22:03 on dist-test-slave-gjrz"
server_key: "84cffceabfa8c4c5c60bf7cbfb38c35b"
server_key_iv: "8f53a40af35ae70f803a162ed72ad3cf"
server_key_version: "encryptionkey@0"
I20260811 06:22:03.125842 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "ee05703f7fd24ff388a962da7cfe6e3f"
format_stamp: "Formatted at 2026-08-11 06:22:03 on dist-test-slave-gjrz"
server_key: "84cffceabfa8c4c5c60bf7cbfb38c35b"
server_key_iv: "8f53a40af35ae70f803a162ed72ad3cf"
server_key_version: "encryptionkey@0"
I20260811 06:22:03.130565 28057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:22:03.133775 28093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:03.134940 28057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:22:03.135099 28057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "ee05703f7fd24ff388a962da7cfe6e3f"
format_stamp: "Formatted at 2026-08-11 06:22:03 on dist-test-slave-gjrz"
server_key: "84cffceabfa8c4c5c60bf7cbfb38c35b"
server_key_iv: "8f53a40af35ae70f803a162ed72ad3cf"
server_key_version: "encryptionkey@0"
I20260811 06:22:03.135324 28057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:03.140487 28057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:03.144028 28057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:03.144304 28057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:03.160521 28057 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42543
I20260811 06:22:03.160573 28189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42543 every 8 connection(s)
I20260811 06:22:03.161779 28057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:03.165148 28190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:03.167791 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28057
I20260811 06:22:03.167975 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:03.168495 23012 external_mini_cluster.cc:1428] Setting key aee5d6c09582eeefec21dde1d112e971
I20260811 06:22:03.177037 28197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:03.170338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54315 (local address 127.0.0.1:42543)
0811 06:22:03.171144 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:03.171159 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:03.171535 (+ 376us) server_negotiation.cc:407] Connection header received
0811 06:22:03.173535 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:03.173562 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:03.173979 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:03.174387 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:03.175313 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:03.175321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:03.175337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:03.175357 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:03.175435 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:03.175460 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:03.175677 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:03.175804 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 06:22:03.175971 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":335,"threads_started":1}
I20260811 06:22:03.177829 28190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f: Bootstrap starting.
I20260811 06:22:03.180665 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:03.181300 28190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:03.182240 28190 log.cc:824] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:03.184887 28190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f: No bootstrap required, opened a new log
I20260811 06:22:03.187711 28197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:03.185159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53131 (local address 127.0.0.1:42543)
0811 06:22:03.185295 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:03.185300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:03.185985 (+ 685us) server_negotiation.cc:407] Connection header received
0811 06:22:03.186140 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:03.186145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:03.186186 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:03.186265 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:03.187074 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:03.187079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:03.187084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:03.187100 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:03.187147 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:03.187161 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:03.187283 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:03.187404 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 06:22:03.187459 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:22:03.192238 28190 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } }
I20260811 06:22:03.192546 28190 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:03.192638 28190 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee05703f7fd24ff388a962da7cfe6e3f, State: Initialized, Role: FOLLOWER
I20260811 06:22:03.193164 28190 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } }
I20260811 06:22:03.193387 28190 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:03.193533 28190 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:03.193703 28190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:03.195737 28190 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } }
I20260811 06:22:03.196182 28190 leader_election.cc:302] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee05703f7fd24ff388a962da7cfe6e3f; no voters:
I20260811 06:22:03.196683 28190 leader_election.cc:288] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:03.196796 28208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:03.197507 28208 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [term 1 LEADER]: Becoming Leader. State: Replica: ee05703f7fd24ff388a962da7cfe6e3f, State: Running, Role: LEADER
I20260811 06:22:03.198068 28208 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } }
I20260811 06:22:03.198565 28190 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:03.200489 28209 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } } }
I20260811 06:22:03.200795 28209 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:03.201774 28210 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee05703f7fd24ff388a962da7cfe6e3f. Latest consensus state: current_term: 1 leader_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee05703f7fd24ff388a962da7cfe6e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42543 } } }
I20260811 06:22:03.202121 28210 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:03.202757 28220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:03.207579 28220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:03.217069 28220 catalog_manager.cc:1347] Generated new cluster ID: 24b5f73de72b426bad5cd57ce0cec24c
I20260811 06:22:03.217171 28220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:03.240164 28220 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:03.240525 28220 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:03.247155 28220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee05703f7fd24ff388a962da7cfe6e3f: Generated new TSK 0
I20260811 06:22:03.247816 28220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:03.259100 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28057
W20260811 06:22:03.260016 28178 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54315
W20260811 06:22:03.260605 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28057: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:03.260725 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28057
2026-08-11T06:22:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1966 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:03 dist-test-slave-gjrz krb5kdc[28247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:03 dist-test-slave-gjrz krb5kdc[28247](info): set up 2 sockets
Aug 11 06:22:03 dist-test-slave-gjrz krb5kdc[28247](info): commencing operation
krb5kdc: starting...
W20260811 06:22:04.511518 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:04 dist-test-slave-gjrz krb5kdc[28247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:04.758823 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:05.081470 28271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:05.081722 28271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:05.081811 28271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:05.088670 28271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:05.088768 28271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:05.088819 28271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:05.088876 28271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:05.088917 28271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:05.098253 28271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:05.099335 28271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:05.100531 28271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:05.111598 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:05.111509 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:05.111515 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:05.112304 28271 server_base.cc:1034] running on GCE node
I20260811 06:22:05.112998 28271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:05.114262 28271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:05.115463 28271 hybrid_clock.cc:648] HybridClock initialized: now 1786429325115407 us; error 41 us; skew 500 ppm
I20260811 06:22:05.115772 28271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:05 dist-test-slave-gjrz krb5kdc[28247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:05.118842 28271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:05.120164 28271 webserver.cc:460] Webserver started at http://127.0.0.1:36189/ using document root <none> and password file <none>
I20260811 06:22:05.120755 28271 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:05.120869 28271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:05.121137 28271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:05.123939 28271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "3906d1c326b84c0e92dcc5b17596f863"
format_stamp: "Formatted at 2026-08-11 06:22:05 on dist-test-slave-gjrz"
server_key: "9a39e974eafdc98b588a6e052edf0e8a"
server_key_iv: "bb6c089b0cde6bb8b821ff6c3b41f989"
server_key_version: "encryptionkey@0"
I20260811 06:22:05.124656 28271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "3906d1c326b84c0e92dcc5b17596f863"
format_stamp: "Formatted at 2026-08-11 06:22:05 on dist-test-slave-gjrz"
server_key: "9a39e974eafdc98b588a6e052edf0e8a"
server_key_iv: "bb6c089b0cde6bb8b821ff6c3b41f989"
server_key_version: "encryptionkey@0"
I20260811 06:22:05.129312 28271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:05.132450 28294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:05.133617 28271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:05.133769 28271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "3906d1c326b84c0e92dcc5b17596f863"
format_stamp: "Formatted at 2026-08-11 06:22:05 on dist-test-slave-gjrz"
server_key: "9a39e974eafdc98b588a6e052edf0e8a"
server_key_iv: "bb6c089b0cde6bb8b821ff6c3b41f989"
server_key_version: "encryptionkey@0"
I20260811 06:22:05.133929 28271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:05.158074 28271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:05.161633 28271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:05.161868 28271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:05.178525 28271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43543
I20260811 06:22:05.178568 28400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43543 every 8 connection(s)
I20260811 06:22:05.179788 28271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:05.183189 28401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:05.187638 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28271
I20260811 06:22:05.187812 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:05.188367 23012 external_mini_cluster.cc:1428] Setting key b013c35ec0d7e3a172a0442f04f524a0
I20260811 06:22:05.193962 28401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863: Bootstrap starting.
I20260811 06:22:05.197358 28401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:05.198474 28401 log.cc:824] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:05.201016 28401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863: No bootstrap required, opened a new log
Aug 11 06:22:05 dist-test-slave-gjrz krb5kdc[28247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:22:05.209600 28401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } }
I20260811 06:22:05.209930 28401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:05.210027 28401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3906d1c326b84c0e92dcc5b17596f863, State: Initialized, Role: FOLLOWER
I20260811 06:22:05.210151 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:05.189938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54135 (local address 127.0.0.1:43543)
0811 06:22:05.190618 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:05.190630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:05.190674 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:22:05.192577 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:05.192605 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:05.192958 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:05.193240 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:05.194988 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:05.196363 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:05.198119 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:05.198825 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:05.199024 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:05.203311 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:05.203320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:05.203338 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:05.203373 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:05.206285 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:05.207110 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:05.207116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:05.207121 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:05.207313 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:05.207702 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:05.207705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:05.207707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:05.208216 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:05.208428 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:05.208857 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 06:22:05.209135 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":305,"threads_started":1}
I20260811 06:22:05.210637 28401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } }
I20260811 06:22:05.210933 28401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:05.211079 28401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:05.211256 28401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:05.213747 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:05.213764 28401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } }
I20260811 06:22:05.214214 28401 leader_election.cc:302] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3906d1c326b84c0e92dcc5b17596f863; no voters:
I20260811 06:22:05.214761 28401 leader_election.cc:288] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:05.214879 28412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:05.215469 28412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [term 1 LEADER]: Becoming Leader. State: Replica: 3906d1c326b84c0e92dcc5b17596f863, State: Running, Role: LEADER
I20260811 06:22:05.216075 28412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } }
I20260811 06:22:05.218665 28416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3906d1c326b84c0e92dcc5b17596f863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } } }
I20260811 06:22:05.219054 28416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:05.223869 28412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3906d1c326b84c0e92dcc5b17596f863. Latest consensus state: current_term: 1 leader_uuid: "3906d1c326b84c0e92dcc5b17596f863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3906d1c326b84c0e92dcc5b17596f863" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43543 } } }
I20260811 06:22:05.224411 28412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:05.228735 28401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:05.229063 28426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:05.234242 28426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:05.245705 28426 catalog_manager.cc:1347] Generated new cluster ID: 255b48ee1c464addb6a994bd203efcff
I20260811 06:22:05.245800 28426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:05.258437 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:05.228626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34301 (local address 127.0.0.1:43543)
0811 06:22:05.228812 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:05.228818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:05.243116 (+ 14298us) server_negotiation.cc:407] Connection header received
0811 06:22:05.243339 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:05.243345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:05.243433 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:05.243532 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:05.244957 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:05.245782 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:05.247526 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:05.248148 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:05.248232 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:05.251034 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:05.251041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:05.251046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:05.251080 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:05.254595 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:05.255465 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:05.255471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:05.255475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:05.255638 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:05.256106 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:05.256111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:05.256114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:05.256515 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:05.256756 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:05.257839 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0811 06:22:05.258074 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 06:22:05.268698 28426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:05.269810 28426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:05.277464 28426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3906d1c326b84c0e92dcc5b17596f863: Generated new TSK 0
I20260811 06:22:05.278141 28426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:05.290292 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28271
I20260811 06:22:05.291230 28396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:05.379856 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28271
2026-08-11T06:22:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:05 dist-test-slave-gjrz krb5kdc[28463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:05 dist-test-slave-gjrz krb5kdc[28463](info): set up 2 sockets
Aug 11 06:22:05 dist-test-slave-gjrz krb5kdc[28463](info): commencing operation
krb5kdc: starting...
W20260811 06:22:06.615715 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.007s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:06 dist-test-slave-gjrz krb5kdc[28463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:06.849377 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:07.164839 28493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:07.165091 28493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:07.165221 28493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:07.172582 28493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:07.172680 28493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:07.172753 28493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:07.172827 28493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:07.172883 28493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:07.182147 28493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:07.183326 28493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:07.184556 28493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:07.195487 28508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:07.195632 28504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:07.195576 28505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:07.196524 28493 server_base.cc:1034] running on GCE node
I20260811 06:22:07.197233 28493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:07.198467 28493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:07.199653 28493 hybrid_clock.cc:648] HybridClock initialized: now 1786429327199623 us; error 32 us; skew 500 ppm
I20260811 06:22:07.199962 28493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:07 dist-test-slave-gjrz krb5kdc[28463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:07.206817 28493 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:07.208128 28493 webserver.cc:460] Webserver started at http://127.0.0.1:46749/ using document root <none> and password file <none>
I20260811 06:22:07.208719 28493 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:07.208829 28493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:07.209107 28493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:07.211906 28493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "175cd82a38584e5fa850a89a18115960"
format_stamp: "Formatted at 2026-08-11 06:22:07 on dist-test-slave-gjrz"
server_key: "37f7f64b3ede870297769b5e5ea2ccc4"
server_key_iv: "dd735e5469b600e0ae68d2c55b2a8d79"
server_key_version: "encryptionkey@0"
I20260811 06:22:07.212631 28493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "175cd82a38584e5fa850a89a18115960"
format_stamp: "Formatted at 2026-08-11 06:22:07 on dist-test-slave-gjrz"
server_key: "37f7f64b3ede870297769b5e5ea2ccc4"
server_key_iv: "dd735e5469b600e0ae68d2c55b2a8d79"
server_key_version: "encryptionkey@0"
I20260811 06:22:07.217300 28493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:07.220496 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:07.221745 28493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:22:07.221896 28493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "175cd82a38584e5fa850a89a18115960"
format_stamp: "Formatted at 2026-08-11 06:22:07 on dist-test-slave-gjrz"
server_key: "37f7f64b3ede870297769b5e5ea2ccc4"
server_key_iv: "dd735e5469b600e0ae68d2c55b2a8d79"
server_key_version: "encryptionkey@0"
I20260811 06:22:07.222074 28493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:07.234642 28493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:07.238584 28493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:07.238822 28493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:07.254989 28493 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44087
I20260811 06:22:07.255012 28649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44087 every 8 connection(s)
I20260811 06:22:07.256273 28493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:07.257933 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28493
I20260811 06:22:07.258098 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:07.258519 23012 external_mini_cluster.cc:1428] Setting key 1ddddc6114f4ad28bd5cb1747488e6ee
I20260811 06:22:07.260221 28650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:07.270903 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:07.260158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54239 (local address 127.0.0.1:44087)
0811 06:22:07.260892 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:07.260903 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:07.260942 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:22:07.262500 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:07.262521 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:07.262885 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:07.263199 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:07.263779 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:07.265085 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:07.266853 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:07.267536 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:07.267660 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:07.268807 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:07.268814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:07.268830 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:07.269355 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:07.269430 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:07.269457 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:07.269557 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:07.269779 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 06:22:07.270036 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":340,"threads_started":1}
I20260811 06:22:07.272369 28650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960: Bootstrap starting.
I20260811 06:22:07.274961 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:07.275812 28650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:07.276747 28650 log.cc:824] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:07.279536 28650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960: No bootstrap required, opened a new log
I20260811 06:22:07.284582 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:07.279058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40463 (local address 127.0.0.1:44087)
0811 06:22:07.279218 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:07.279223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:07.279635 (+ 412us) server_negotiation.cc:407] Connection header received
0811 06:22:07.279805 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:07.279809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:07.279861 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:07.279959 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:07.280409 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:07.281100 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:07.282384 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:07.282824 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:07.282930 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:07.283899 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:07.283903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:07.283909 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:07.283927 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:07.283973 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:07.283989 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:07.284073 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:07.284208 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 06:22:07.284292 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:22:07.286914 28650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260811 06:22:07.287256 28650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:07.287395 28650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175cd82a38584e5fa850a89a18115960, State: Initialized, Role: FOLLOWER
I20260811 06:22:07.287992 28650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260811 06:22:07.288193 28650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:07.288290 28650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:07.288410 28650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:07.290555 28650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260811 06:22:07.291080 28650 leader_election.cc:302] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 175cd82a38584e5fa850a89a18115960; no voters:
I20260811 06:22:07.291603 28650 leader_election.cc:288] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:07.291692 28665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:07.292285 28665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [term 1 LEADER]: Becoming Leader. State: Replica: 175cd82a38584e5fa850a89a18115960, State: Running, Role: LEADER
I20260811 06:22:07.292826 28665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260811 06:22:07.293411 28650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:07.295359 28667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 175cd82a38584e5fa850a89a18115960. Latest consensus state: current_term: 1 leader_uuid: "175cd82a38584e5fa850a89a18115960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } } }
I20260811 06:22:07.295785 28667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:07.295748 28666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "175cd82a38584e5fa850a89a18115960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cd82a38584e5fa850a89a18115960" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } } }
I20260811 06:22:07.296057 28666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:07.298206 28679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:07.302953 28679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:07.311203 28679 catalog_manager.cc:1347] Generated new cluster ID: 3cc9f247fb46458dbe93bd82c40d6060
I20260811 06:22:07.311308 28679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:07.330528 28679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:07.331743 28679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:07.342281 28679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 175cd82a38584e5fa850a89a18115960: Generated new TSK 0
I20260811 06:22:07.342947 28679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:07.354045 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28493
W20260811 06:22:07.355026 28634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54239
W20260811 06:22:07.355645 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28493: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:07.355794 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28493
2026-08-11T06:22:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1982 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:07 dist-test-slave-gjrz krb5kdc[28697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:07 dist-test-slave-gjrz krb5kdc[28697](info): set up 2 sockets
Aug 11 06:22:07 dist-test-slave-gjrz krb5kdc[28697](info): commencing operation
krb5kdc: starting...
W20260811 06:22:08.605957 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:08 dist-test-slave-gjrz krb5kdc[28697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:08.842387 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:09.155143 28725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:09.155421 28725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:09.155562 28725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:09.162374 28725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:09.162485 28725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:09.162565 28725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:09.162650 28725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:09.162710 28725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:09.171833 28725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:09.172932 28725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:09.174129 28725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:09.185165 28744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:09.185127 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:09.185149 28745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:09.186002 28725 server_base.cc:1034] running on GCE node
I20260811 06:22:09.186647 28725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:09.187879 28725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:09.189062 28725 hybrid_clock.cc:648] HybridClock initialized: now 1786429329189017 us; error 38 us; skew 500 ppm
I20260811 06:22:09.189342 28725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:09 dist-test-slave-gjrz krb5kdc[28697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:09.192430 28725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:09.193753 28725 webserver.cc:460] Webserver started at http://127.0.0.1:36921/ using document root <none> and password file <none>
I20260811 06:22:09.194351 28725 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:09.194463 28725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:09.194733 28725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:09.197543 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "1f37e121731448c084c479192553a5e6"
format_stamp: "Formatted at 2026-08-11 06:22:09 on dist-test-slave-gjrz"
server_key: "3b51d247480fb0a7ef60ca9d6ec39e7a"
server_key_iv: "fb4b6fd7fc19e3e7827bc721885f3c06"
server_key_version: "encryptionkey@0"
I20260811 06:22:09.198256 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "1f37e121731448c084c479192553a5e6"
format_stamp: "Formatted at 2026-08-11 06:22:09 on dist-test-slave-gjrz"
server_key: "3b51d247480fb0a7ef60ca9d6ec39e7a"
server_key_iv: "fb4b6fd7fc19e3e7827bc721885f3c06"
server_key_version: "encryptionkey@0"
I20260811 06:22:09.202960 28725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:22:09.206512 28767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:09.207702 28725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:09.207850 28725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "1f37e121731448c084c479192553a5e6"
format_stamp: "Formatted at 2026-08-11 06:22:09 on dist-test-slave-gjrz"
server_key: "3b51d247480fb0a7ef60ca9d6ec39e7a"
server_key_iv: "fb4b6fd7fc19e3e7827bc721885f3c06"
server_key_version: "encryptionkey@0"
I20260811 06:22:09.208012 28725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:09.213132 28725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:09.216635 28725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:09.216883 28725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:09.233239 28725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33647
I20260811 06:22:09.233294 28856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33647 every 8 connection(s)
I20260811 06:22:09.234432 28725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:09.237753 28859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:09.241488 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28725
I20260811 06:22:09.241645 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:09.242086 23012 external_mini_cluster.cc:1428] Setting key 117bf86d62259a8dc54ae0b744e9b450
I20260811 06:22:09.249143 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6: Bootstrap starting.
I20260811 06:22:09.250924 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:09.243721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36529 (local address 127.0.0.1:33647)
0811 06:22:09.244503 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:09.244518 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:09.244567 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:22:09.247417 (+ 2850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:09.247443 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:09.247842 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:09.248223 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:09.249113 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:09.249122 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:09.249142 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:09.249161 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:09.249230 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:09.249263 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:09.249466 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:09.249626 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 06:22:09.249792 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":328,"threads_started":1}
I20260811 06:22:09.252640 28859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:09.253806 28859 log.cc:824] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:09.254439 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:09.256362 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6: No bootstrap required, opened a new log
I20260811 06:22:09.260685 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:09.258628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40715 (local address 127.0.0.1:33647)
0811 06:22:09.258783 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:09.258788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:09.259053 (+ 265us) server_negotiation.cc:407] Connection header received
0811 06:22:09.259171 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:09.259176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:09.259224 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:09.259352 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:09.260081 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:09.260086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:09.260090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:09.260108 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:09.260153 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:09.260171 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:09.260288 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:09.260340 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:22:09.260387 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:22:09.264189 28859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260811 06:22:09.264493 28859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:09.264564 28859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f37e121731448c084c479192553a5e6, State: Initialized, Role: FOLLOWER
I20260811 06:22:09.265004 28859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260811 06:22:09.265213 28859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:09.265313 28859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:09.265424 28859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:09.267371 28859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260811 06:22:09.267863 28859 leader_election.cc:302] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f37e121731448c084c479192553a5e6; no voters:
I20260811 06:22:09.268327 28859 leader_election.cc:288] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:09.268441 28874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:09.269034 28874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [term 1 LEADER]: Becoming Leader. State: Replica: 1f37e121731448c084c479192553a5e6, State: Running, Role: LEADER
I20260811 06:22:09.269526 28874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260811 06:22:09.270071 28859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:09.272019 28880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f37e121731448c084c479192553a5e6. Latest consensus state: current_term: 1 leader_uuid: "1f37e121731448c084c479192553a5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } } }
I20260811 06:22:09.272404 28880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:09.273126 28879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f37e121731448c084c479192553a5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f37e121731448c084c479192553a5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } } }
I20260811 06:22:09.273473 28879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:09.273976 28893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:09.278918 28893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:09.287169 28893 catalog_manager.cc:1347] Generated new cluster ID: 30cbf7086cab44408de3a645e62fa9db
I20260811 06:22:09.287289 28893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:09.304987 28893 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:09.305377 28893 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:09.312168 28893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f37e121731448c084c479192553a5e6: Generated new TSK 0
I20260811 06:22:09.312999 28893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:09.326566 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28725
W20260811 06:22:09.327622 28852 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36529
W20260811 06:22:09.328274 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28725: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:09.328390 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28725
2026-08-11T06:22:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1966 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:09 dist-test-slave-gjrz krb5kdc[28918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:09 dist-test-slave-gjrz krb5kdc[28918](info): set up 2 sockets
Aug 11 06:22:09 dist-test-slave-gjrz krb5kdc[28918](info): commencing operation
krb5kdc: starting...
W20260811 06:22:10.571403 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:10 dist-test-slave-gjrz krb5kdc[28918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:10.810065 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41769
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:11.125404 28947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:11.125655 28947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:11.125743 28947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:11.132715 28947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:11.132813 28947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:11.132860 28947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:11.132917 28947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:11.132967 28947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:11.142231 28947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:11.143388 28947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:11.144577 28947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:11.155529 28970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:11.155603 28964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:11.155582 28968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:11.156418 28947 server_base.cc:1034] running on GCE node
I20260811 06:22:11.157145 28947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:11.158378 28947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:11.159579 28947 hybrid_clock.cc:648] HybridClock initialized: now 1786429331159536 us; error 31 us; skew 500 ppm
I20260811 06:22:11.159901 28947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:11 dist-test-slave-gjrz krb5kdc[28918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:11.162835 28947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:11.164209 28947 webserver.cc:460] Webserver started at http://127.0.0.1:36581/ using document root <none> and password file <none>
I20260811 06:22:11.164790 28947 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:11.164911 28947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:11.165253 28947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:11.168058 28947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "369127e92a48489f9362398414c0a357"
format_stamp: "Formatted at 2026-08-11 06:22:11 on dist-test-slave-gjrz"
server_key: "75a872d88d024b0553f95f01e96b196d"
server_key_iv: "6fc93e0377a91426cc3b8ecf24f38a39"
server_key_version: "encryptionkey@0"
I20260811 06:22:11.168773 28947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "369127e92a48489f9362398414c0a357"
format_stamp: "Formatted at 2026-08-11 06:22:11 on dist-test-slave-gjrz"
server_key: "75a872d88d024b0553f95f01e96b196d"
server_key_iv: "6fc93e0377a91426cc3b8ecf24f38a39"
server_key_version: "encryptionkey@0"
I20260811 06:22:11.173583 28947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 06:22:11.176865 28987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:11.178041 28947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:22:11.178202 28947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "369127e92a48489f9362398414c0a357"
format_stamp: "Formatted at 2026-08-11 06:22:11 on dist-test-slave-gjrz"
server_key: "75a872d88d024b0553f95f01e96b196d"
server_key_iv: "6fc93e0377a91426cc3b8ecf24f38a39"
server_key_version: "encryptionkey@0"
I20260811 06:22:11.178367 28947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:11.193611 28947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:11.197640 28947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:11.197886 28947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:11.214766 28947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41769
I20260811 06:22:11.214807 29096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41769 every 8 connection(s)
I20260811 06:22:11.215971 28947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:11.219326 29097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:11.220594 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 28947
I20260811 06:22:11.220764 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:11.221308 23012 external_mini_cluster.cc:1428] Setting key 5f8258f2a728612f79d3752bc3413347
I20260811 06:22:11.232653 29097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: Bootstrap starting.
Aug 11 06:22:11 dist-test-slave-gjrz krb5kdc[28918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:22:11.236223 29097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:11.237356 29097 log.cc:824] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:11.240216 29097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: No bootstrap required, opened a new log
I20260811 06:22:11.243468 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:11.223029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52517 (local address 127.0.0.1:41769)
0811 06:22:11.223806 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:11.223817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:11.223859 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:22:11.225466 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:11.225486 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:11.225847 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:11.226148 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:11.227892 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:11.229176 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:11.230897 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:11.231601 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:11.231724 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:11.236416 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:11.236427 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:11.236441 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:11.236479 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:11.239529 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:11.240351 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:11.240357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:11.240363 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:11.240515 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:11.241002 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:11.241005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:11.241007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:11.241482 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:11.241709 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:11.242224 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 06:22:11.242485 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":370,"threads_started":1}
I20260811 06:22:11.246860 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:11.248687 29097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } }
I20260811 06:22:11.249063 29097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:11.249162 29097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369127e92a48489f9362398414c0a357, State: Initialized, Role: FOLLOWER
I20260811 06:22:11.249773 29097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } }
I20260811 06:22:11.250066 29097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:11.250202 29097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:11.250382 29097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:11.253013 29097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } }
I20260811 06:22:11.253441 29097 leader_election.cc:302] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369127e92a48489f9362398414c0a357; no voters:
I20260811 06:22:11.253943 29097 leader_election.cc:288] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:11.254112 29120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:11.256045 29097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:11.262115 29120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [term 1 LEADER]: Becoming Leader. State: Replica: 369127e92a48489f9362398414c0a357, State: Running, Role: LEADER
I20260811 06:22:11.262714 29120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } }
I20260811 06:22:11.265117 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:11.252003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:38381 (local address 127.0.0.1:41769)
0811 06:22:11.252170 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:11.252176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:11.252590 (+ 414us) server_negotiation.cc:407] Connection header received
0811 06:22:11.252711 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:11.252716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:11.252786 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:11.252883 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:11.254377 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:11.255245 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:11.256486 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:11.256947 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:11.257056 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:11.259648 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:11.259654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:11.259659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:11.259689 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:11.262309 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:11.262963 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:11.262967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:11.262971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:11.263099 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:11.263628 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:11.263634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:11.263637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:11.264044 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:11.264285 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:11.264636 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 06:22:11.264790 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:22:11.268628 29121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "369127e92a48489f9362398414c0a357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } } }
I20260811 06:22:11.268778 29123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369127e92a48489f9362398414c0a357. Latest consensus state: current_term: 1 leader_uuid: "369127e92a48489f9362398414c0a357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369127e92a48489f9362398414c0a357" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41769 } } }
I20260811 06:22:11.268989 29121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:11.269064 29123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357 [sys.catalog]: This master's current role is: LEADER
W20260811 06:22:11.272681 29147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:22:11.272782 29147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:22:11.272919 29148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:11.274434 29148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:11.281503 29148 catalog_manager.cc:1347] Generated new cluster ID: 1a06dcde1f2843cbb2ebe541765d2b99
I20260811 06:22:11.281599 29148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:11.306619 29148 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:11.307762 29148 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:11.316345 29148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 369127e92a48489f9362398414c0a357: Generated new TSK 0
I20260811 06:22:11.317274 29148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:11.328994 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 28947
I20260811 06:22:11.330348 29091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:11.420276 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 28947
W20260811 06:22:11.443130 28914 connection.cc:535] client connection to 127.0.0.1:41769 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41769): Connection reset by peer (error 104)
2026-08-11T06:22:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2090 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:11 dist-test-slave-gjrz krb5kdc[29161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:11 dist-test-slave-gjrz krb5kdc[29161](info): set up 2 sockets
Aug 11 06:22:11 dist-test-slave-gjrz krb5kdc[29161](info): commencing operation
krb5kdc: starting...
W20260811 06:22:12.656818 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:12 dist-test-slave-gjrz krb5kdc[29161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:12.900484 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:13.219281 29184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:13.219560 29184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:13.219697 29184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:13.226954 29184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:13.227073 29184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:13.227151 29184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:13.227240 29184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:13.227321 29184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:13.236625 29184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:13.237769 29184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:13.238982 29184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:13.249974 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:13.249910 29196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:13.249861 29200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:13.250979 29184 server_base.cc:1034] running on GCE node
I20260811 06:22:13.251718 29184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:13.252955 29184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:13.254172 29184 hybrid_clock.cc:648] HybridClock initialized: now 1786429333254114 us; error 44 us; skew 500 ppm
I20260811 06:22:13.254484 29184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:13 dist-test-slave-gjrz krb5kdc[29161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:13.261025 29184 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:13.262444 29184 webserver.cc:460] Webserver started at http://127.0.0.1:42303/ using document root <none> and password file <none>
I20260811 06:22:13.263058 29184 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:13.263172 29184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:13.263478 29184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:13.266268 29184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "d71e12b7ddae40f4912e84d9dc1fbac3"
format_stamp: "Formatted at 2026-08-11 06:22:13 on dist-test-slave-gjrz"
server_key: "dd5b92ab86bb7a107d344f5f1a0c9fe0"
server_key_iv: "af74042bf1178ebf3f54e4088e4ca6d9"
server_key_version: "encryptionkey@0"
I20260811 06:22:13.267016 29184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "d71e12b7ddae40f4912e84d9dc1fbac3"
format_stamp: "Formatted at 2026-08-11 06:22:13 on dist-test-slave-gjrz"
server_key: "dd5b92ab86bb7a107d344f5f1a0c9fe0"
server_key_iv: "af74042bf1178ebf3f54e4088e4ca6d9"
server_key_version: "encryptionkey@0"
I20260811 06:22:13.271770 29184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:22:13.274940 29216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:13.276131 29184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:13.276283 29184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "d71e12b7ddae40f4912e84d9dc1fbac3"
format_stamp: "Formatted at 2026-08-11 06:22:13 on dist-test-slave-gjrz"
server_key: "dd5b92ab86bb7a107d344f5f1a0c9fe0"
server_key_iv: "af74042bf1178ebf3f54e4088e4ca6d9"
server_key_version: "encryptionkey@0"
I20260811 06:22:13.276445 29184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:13.301343 29184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:13.305379 29184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:13.305641 29184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:13.322727 29184 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42961
I20260811 06:22:13.322758 29333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42961 every 8 connection(s)
I20260811 06:22:13.324015 29184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:13.327517 29334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:13.330404 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29184
I20260811 06:22:13.330583 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:13.331151 23012 external_mini_cluster.cc:1428] Setting key f771b881ac91503a571e65753026b5ca
I20260811 06:22:13.338945 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3: Bootstrap starting.
I20260811 06:22:13.342315 29334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:13.343436 29334 log.cc:824] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:13.343819 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:13.332886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:41243 (local address 127.0.0.1:42961)
0811 06:22:13.333632 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:13.333643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:13.333683 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:22:13.335351 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:13.335373 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:13.335758 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:13.336062 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:13.336599 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:13.337879 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:13.339691 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:13.340336 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:13.340472 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:13.341663 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:13.341670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:13.341682 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:13.342184 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:13.342263 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:13.342287 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:13.342428 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:13.342680 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 06:22:13.342853 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":323,"threads_started":1}
I20260811 06:22:13.346050 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3: No bootstrap required, opened a new log
I20260811 06:22:13.347415 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:13.355461 29334 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 06:22:13.355875 29334 raft_consensus.cc:374] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:13.355970 29334 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71e12b7ddae40f4912e84d9dc1fbac3, State: Initialized, Role: FOLLOWER
I20260811 06:22:13.356561 29334 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 06:22:13.356819 29334 raft_consensus.cc:388] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:13.356947 29334 raft_consensus.cc:482] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:13.357142 29334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:13.358788 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:13.352547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:32957 (local address 127.0.0.1:42961)
0811 06:22:13.352728 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:13.352734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:13.353076 (+ 342us) server_negotiation.cc:407] Connection header received
0811 06:22:13.353242 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:13.353245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:13.353307 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:13.353390 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:13.353905 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:13.354637 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:13.356432 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:13.356896 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:13.357043 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:13.358102 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:13.358107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:13.358110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:13.358128 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:13.358163 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:13.358177 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:13.358261 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:13.358435 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 06:22:13.358473 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:22:13.359737 29334 raft_consensus.cc:504] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 06:22:13.360172 29334 leader_election.cc:302] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d71e12b7ddae40f4912e84d9dc1fbac3; no voters:
I20260811 06:22:13.360683 29334 leader_election.cc:288] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:13.360798 29356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:13.361807 29356 raft_consensus.cc:686] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [term 1 LEADER]: Becoming Leader. State: Replica: d71e12b7ddae40f4912e84d9dc1fbac3, State: Running, Role: LEADER
I20260811 06:22:13.362380 29356 consensus_queue.cc:237] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260811 06:22:13.362692 29334 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:13.364851 29358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d71e12b7ddae40f4912e84d9dc1fbac3. Latest consensus state: current_term: 1 leader_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260811 06:22:13.365167 29358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:13.364933 29357 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e12b7ddae40f4912e84d9dc1fbac3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260811 06:22:13.365278 29357 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:13.365972 29372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:13.374780 29372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:13.382117 29372 catalog_manager.cc:1347] Generated new cluster ID: aa2fdf4bb4184cd89affff390e7c180e
I20260811 06:22:13.382210 29372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:13.389734 29372 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:13.391410 29372 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:13.400002 29372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d71e12b7ddae40f4912e84d9dc1fbac3: Generated new TSK 0
I20260811 06:22:13.400854 29372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:13.414249 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29184
W20260811 06:22:13.415290 29322 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.164:41243
W20260811 06:22:13.415938 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29184: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:13.416088 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29184
W20260811 06:22:13.438993 29157 connection.cc:535] client connection to 127.0.0.1:42961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42961): Connection reset by peer (error 104)
2026-08-11T06:22:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1995 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:13 dist-test-slave-gjrz krb5kdc[29396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:13 dist-test-slave-gjrz krb5kdc[29396](info): set up 2 sockets
Aug 11 06:22:13 dist-test-slave-gjrz krb5kdc[29396](info): commencing operation
krb5kdc: starting...
W20260811 06:22:14.670751 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:14 dist-test-slave-gjrz krb5kdc[29396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:14.940254 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:15.260376 29422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:15.260694 29422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:15.260829 29422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:15.267825 29422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:15.267931 29422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:15.268010 29422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:15.268091 29422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:15.268149 29422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:15.277498 29422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:15.278652 29422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:15.279914 29422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:15.291287 29429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:15.291410 29428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:15.291316 29443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:15.292151 29422 server_base.cc:1034] running on GCE node
I20260811 06:22:15.292867 29422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:15.294103 29422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:15.295334 29422 hybrid_clock.cc:648] HybridClock initialized: now 1786429335295283 us; error 37 us; skew 500 ppm
I20260811 06:22:15.295652 29422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:15 dist-test-slave-gjrz krb5kdc[29396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:15.298559 29422 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:15.299954 29422 webserver.cc:460] Webserver started at http://127.0.0.1:35143/ using document root <none> and password file <none>
I20260811 06:22:15.300570 29422 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:15.300691 29422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:15.300969 29422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:15.303887 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "a748eaadddfa4a60a3a3228627238c33"
format_stamp: "Formatted at 2026-08-11 06:22:15 on dist-test-slave-gjrz"
server_key: "8f526de43a874b2aacab1acda9866881"
server_key_iv: "0deaf3f9de7b2c215322e5b186ef77b1"
server_key_version: "encryptionkey@0"
I20260811 06:22:15.304636 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "a748eaadddfa4a60a3a3228627238c33"
format_stamp: "Formatted at 2026-08-11 06:22:15 on dist-test-slave-gjrz"
server_key: "8f526de43a874b2aacab1acda9866881"
server_key_iv: "0deaf3f9de7b2c215322e5b186ef77b1"
server_key_version: "encryptionkey@0"
I20260811 06:22:15.309378 29422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 06:22:15.312548 29455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:15.313697 29422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:22:15.313853 29422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "a748eaadddfa4a60a3a3228627238c33"
format_stamp: "Formatted at 2026-08-11 06:22:15 on dist-test-slave-gjrz"
server_key: "8f526de43a874b2aacab1acda9866881"
server_key_iv: "0deaf3f9de7b2c215322e5b186ef77b1"
server_key_version: "encryptionkey@0"
I20260811 06:22:15.314040 29422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:15.319430 29422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:15.322969 29422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:15.323235 29422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:15.339939 29422 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42935
I20260811 06:22:15.339991 29557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42935 every 8 connection(s)
I20260811 06:22:15.341166 29422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:15.341825 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29422
I20260811 06:22:15.341996 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:15.342542 23012 external_mini_cluster.cc:1428] Setting key a57847ce10ad6100868130e783ac42ab
I20260811 06:22:15.345198 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:15.349603 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:15.344175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:40975 (local address 127.0.0.1:42935)
0811 06:22:15.344898 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:15.344909 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:15.344945 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:22:15.346542 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:15.346563 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:15.346864 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:15.347190 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:15.348133 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:15.348139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:15.348152 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:15.348165 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:15.348238 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:15.348259 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:15.348470 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:15.348624 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 06:22:15.348788 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":307,"threads_started":1}
I20260811 06:22:15.353031 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:15.356166 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33: Bootstrap starting.
I20260811 06:22:15.359498 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:15.357169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:41431 (local address 127.0.0.1:42935)
0811 06:22:15.357322 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:15.357327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:15.357773 (+ 446us) server_negotiation.cc:407] Connection header received
0811 06:22:15.357921 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:15.357924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:15.357967 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:15.358042 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:15.358900 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:15.358904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:15.358907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:15.358920 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:15.358956 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:15.358969 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:15.359047 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:15.359181 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 06:22:15.359222 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:22:15.359830 29560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:15.360909 29560 log.cc:824] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:15.363241 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33: No bootstrap required, opened a new log
I20260811 06:22:15.370306 29560 raft_consensus.cc:348] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } }
I20260811 06:22:15.370620 29560 raft_consensus.cc:374] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:15.370692 29560 raft_consensus.cc:729] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a748eaadddfa4a60a3a3228627238c33, State: Initialized, Role: FOLLOWER
I20260811 06:22:15.371137 29560 consensus_queue.cc:260] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } }
I20260811 06:22:15.371358 29560 raft_consensus.cc:388] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:15.371466 29560 raft_consensus.cc:482] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:15.371601 29560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:15.373566 29560 raft_consensus.cc:504] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } }
I20260811 06:22:15.373973 29560 leader_election.cc:302] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a748eaadddfa4a60a3a3228627238c33; no voters:
I20260811 06:22:15.374552 29560 leader_election.cc:288] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:15.374665 29573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:15.375250 29573 raft_consensus.cc:686] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [term 1 LEADER]: Becoming Leader. State: Replica: a748eaadddfa4a60a3a3228627238c33, State: Running, Role: LEADER
I20260811 06:22:15.375747 29573 consensus_queue.cc:237] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } }
I20260811 06:22:15.376400 29560 sys_catalog.cc:564] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:15.378237 29575 sys_catalog.cc:455] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a748eaadddfa4a60a3a3228627238c33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } } }
I20260811 06:22:15.378630 29575 sys_catalog.cc:458] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:15.378567 29576 sys_catalog.cc:455] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a748eaadddfa4a60a3a3228627238c33. Latest consensus state: current_term: 1 leader_uuid: "a748eaadddfa4a60a3a3228627238c33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a748eaadddfa4a60a3a3228627238c33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42935 } } }
I20260811 06:22:15.378825 29576 sys_catalog.cc:458] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:15.384086 29595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:15.388955 29595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:15.396749 29595 catalog_manager.cc:1347] Generated new cluster ID: 66329de6d32d4998a979a9800d405ca1
I20260811 06:22:15.396848 29595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:15.419822 29595 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:15.420181 29595 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:15.430567 29595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a748eaadddfa4a60a3a3228627238c33: Generated new TSK 0
I20260811 06:22:15.431458 29595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:15.448485 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29422
W20260811 06:22:15.449532 29552 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.164:40975
W20260811 06:22:15.450212 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29422: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:15.450345 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29422
2026-08-11T06:22:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2039 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:15 dist-test-slave-gjrz krb5kdc[29623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:15 dist-test-slave-gjrz krb5kdc[29623](info): set up 2 sockets
Aug 11 06:22:15 dist-test-slave-gjrz krb5kdc[29623](info): commencing operation
krb5kdc: starting...
W20260811 06:22:16.698107 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:16 dist-test-slave-gjrz krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:16.940521 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:17.257550 29645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:17.257822 29645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:17.257961 29645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:17.264909 29645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:17.265017 29645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:17.265094 29645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:17.265185 29645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:17.265246 29645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:17.274549 29645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:17.275704 29645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:17.276907 29645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:17.287819 29656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:17.288033 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:17.288041 29658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:17.288780 29645 server_base.cc:1034] running on GCE node
I20260811 06:22:17.289604 29645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:17.290982 29645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:17.292191 29645 hybrid_clock.cc:648] HybridClock initialized: now 1786429337292147 us; error 28 us; skew 500 ppm
I20260811 06:22:17.292537 29645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:17 dist-test-slave-gjrz krb5kdc[29623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:17.295544 29645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:17.296837 29645 webserver.cc:460] Webserver started at http://127.0.0.1:37281/ using document root <none> and password file <none>
I20260811 06:22:17.297487 29645 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:17.297616 29645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:17.297925 29645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:17.300825 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "43e15ac520d84ab69ff821d537e1c8a9"
format_stamp: "Formatted at 2026-08-11 06:22:17 on dist-test-slave-gjrz"
server_key: "f0509c952ca2b7d368357d408e17c11c"
server_key_iv: "ec97fda0df5aabe486af2c915a2ce72e"
server_key_version: "encryptionkey@0"
I20260811 06:22:17.301709 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "43e15ac520d84ab69ff821d537e1c8a9"
format_stamp: "Formatted at 2026-08-11 06:22:17 on dist-test-slave-gjrz"
server_key: "f0509c952ca2b7d368357d408e17c11c"
server_key_iv: "ec97fda0df5aabe486af2c915a2ce72e"
server_key_version: "encryptionkey@0"
I20260811 06:22:17.306735 29645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:22:17.309978 29679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:17.311157 29645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:22:17.311347 29645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "43e15ac520d84ab69ff821d537e1c8a9"
format_stamp: "Formatted at 2026-08-11 06:22:17 on dist-test-slave-gjrz"
server_key: "f0509c952ca2b7d368357d408e17c11c"
server_key_iv: "ec97fda0df5aabe486af2c915a2ce72e"
server_key_version: "encryptionkey@0"
I20260811 06:22:17.311513 29645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:17.337699 29645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:17.341732 29645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:17.342031 29645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:17.359030 29645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35365
I20260811 06:22:17.359061 29775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35365 every 8 connection(s)
I20260811 06:22:17.360323 29645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:17.363863 29779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:17.370234 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29645
I20260811 06:22:17.370427 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:17.370960 23012 external_mini_cluster.cc:1428] Setting key da7ab6bf06889df9421f576aa43deb36
I20260811 06:22:17.374708 29779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9: Bootstrap starting.
I20260811 06:22:17.378068 29779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:17.379189 29779 log.cc:824] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:17.382205 29779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9: No bootstrap required, opened a new log
Aug 11 06:22:17 dist-test-slave-gjrz krb5kdc[29623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:22:17.390419 29779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 06:22:17.390740 29779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:17.390833 29779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43e15ac520d84ab69ff821d537e1c8a9, State: Initialized, Role: FOLLOWER
I20260811 06:22:17.391436 29779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 06:22:17.391660 29779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:17.391798 29779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:17.391989 29779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:17.394784 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:17.372835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:35291 (local address 127.0.0.1:35365)
0811 06:22:17.373686 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:17.373702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:17.374012 (+ 310us) server_negotiation.cc:407] Connection header received
0811 06:22:17.376002 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:17.376053 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:17.376431 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:17.376752 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:17.378545 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:17.380199 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:17.381924 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:17.382607 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:17.382737 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:17.387351 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:17.387361 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:17.387376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:17.387413 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:17.390305 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:17.391136 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:17.391142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:17.391147 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:17.391351 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:17.391828 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:17.391833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:17.391836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:17.392532 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:17.392863 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:17.393228 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 06:22:17.393563 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":358,"threads_started":1}
I20260811 06:22:17.395046 29779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 06:22:17.395612 29779 leader_election.cc:302] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43e15ac520d84ab69ff821d537e1c8a9; no voters:
I20260811 06:22:17.396239 29779 leader_election.cc:288] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:17.396380 29785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:17.397611 29785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [term 1 LEADER]: Becoming Leader. State: Replica: 43e15ac520d84ab69ff821d537e1c8a9, State: Running, Role: LEADER
I20260811 06:22:17.398315 29785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 06:22:17.398566 29779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:17.399194 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:17.401081 29791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43e15ac520d84ab69ff821d537e1c8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 06:22:17.401429 29791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:17.401938 29793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e15ac520d84ab69ff821d537e1c8a9. Latest consensus state: current_term: 1 leader_uuid: "43e15ac520d84ab69ff821d537e1c8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e15ac520d84ab69ff821d537e1c8a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 06:22:17.402244 29793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:17.404313 29800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:17.411211 29800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:17.420034 29800 catalog_manager.cc:1347] Generated new cluster ID: ff28ded0f47f49c595795ac69aad2f64
I20260811 06:22:17.420133 29800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:17.446277 29800 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:17.448063 29800 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:17.456297 29800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43e15ac520d84ab69ff821d537e1c8a9: Generated new TSK 0
I20260811 06:22:17.457273 29800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:17.460450 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:17.436728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:58471 (local address 127.0.0.1:35365)
0811 06:22:17.436909 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:17.436915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:17.443436 (+ 6521us) server_negotiation.cc:407] Connection header received
0811 06:22:17.443600 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:17.443606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:17.443691 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:17.443797 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:17.445846 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:17.447030 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:17.448996 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:17.449730 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:17.449831 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:17.453142 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:17.453149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:17.453157 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:17.453193 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:17.456841 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:17.457909 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:17.457915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:17.457918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:17.458104 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:17.458693 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:17.458698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:17.458701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:17.459280 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:17.459543 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:17.459898 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 06:22:17.460092 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:22:17.464588 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29645
I20260811 06:22:17.465998 29771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:17.556708 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29645
W20260811 06:22:17.577879 29620 connection.cc:535] client connection to 127.0.0.1:35365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35365): Connection reset by peer (error 104)
2026-08-11T06:22:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:17 dist-test-slave-gjrz krb5kdc[29837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:17 dist-test-slave-gjrz krb5kdc[29837](info): set up 2 sockets
Aug 11 06:22:17 dist-test-slave-gjrz krb5kdc[29837](info): commencing operation
krb5kdc: starting...
W20260811 06:22:18.802119 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:18 dist-test-slave-gjrz krb5kdc[29837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:19.054659 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:19.373908 29872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:19.374181 29872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:19.374320 29872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:19.381240 29872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:19.381354 29872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:19.381428 29872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:19.381511 29872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:19.381569 29872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:19.390754 29872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:19.391862 29872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:19.393059 29872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:19.403945 29896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:19.404016 29895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:19.403908 29900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:19.404585 29872 server_base.cc:1034] running on GCE node
I20260811 06:22:19.405279 29872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:19.406497 29872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:19.407683 29872 hybrid_clock.cc:648] HybridClock initialized: now 1786429339407639 us; error 35 us; skew 500 ppm
I20260811 06:22:19.407981 29872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:19 dist-test-slave-gjrz krb5kdc[29837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:19.414597 29872 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:19.415974 29872 webserver.cc:460] Webserver started at http://127.0.0.1:33683/ using document root <none> and password file <none>
I20260811 06:22:19.416587 29872 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:19.416702 29872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:19.416970 29872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:19.419746 29872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "2cb0b58daf4d42a7ba12ac81b3c53580"
format_stamp: "Formatted at 2026-08-11 06:22:19 on dist-test-slave-gjrz"
server_key: "7e2a8849265dbf7507af3f645ea69df6"
server_key_iv: "35c483173cd45f903a32aab0ba68fc63"
server_key_version: "encryptionkey@0"
I20260811 06:22:19.420450 29872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "2cb0b58daf4d42a7ba12ac81b3c53580"
format_stamp: "Formatted at 2026-08-11 06:22:19 on dist-test-slave-gjrz"
server_key: "7e2a8849265dbf7507af3f645ea69df6"
server_key_iv: "35c483173cd45f903a32aab0ba68fc63"
server_key_version: "encryptionkey@0"
I20260811 06:22:19.425153 29872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:19.428237 29910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:19.429373 29872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:22:19.429533 29872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "2cb0b58daf4d42a7ba12ac81b3c53580"
format_stamp: "Formatted at 2026-08-11 06:22:19 on dist-test-slave-gjrz"
server_key: "7e2a8849265dbf7507af3f645ea69df6"
server_key_iv: "35c483173cd45f903a32aab0ba68fc63"
server_key_version: "encryptionkey@0"
I20260811 06:22:19.429697 29872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:19.451318 29872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:19.454903 29872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:19.455158 29872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:19.471887 29872 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45815
I20260811 06:22:19.471920 30002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45815 every 8 connection(s)
I20260811 06:22:19.473119 29872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:19.475724 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 29872
I20260811 06:22:19.475873 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:19.476292 23012 external_mini_cluster.cc:1428] Setting key 5400a2630c77955f2d85154e748cb7dc
I20260811 06:22:19.476621 30006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:19.489495 30006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580: Bootstrap starting.
I20260811 06:22:19.489586 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:19.477886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:38707 (local address 127.0.0.1:45815)
0811 06:22:19.478594 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:19.478604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:19.478643 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:22:19.481039 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:19.481060 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:19.481466 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:19.481769 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:19.482355 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:19.483732 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:19.485539 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:19.486181 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:19.486310 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:19.487556 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:19.487564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:19.487580 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:19.488066 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:19.488153 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:19.488176 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:19.488285 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:19.488539 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 06:22:19.488697 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
I20260811 06:22:19.493218 30006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:19.493952 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:19.494482 30006 log.cc:824] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:19.497742 30006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580: No bootstrap required, opened a new log
I20260811 06:22:19.505128 30006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } }
I20260811 06:22:19.505470 30006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:19.505571 30006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb0b58daf4d42a7ba12ac81b3c53580, State: Initialized, Role: FOLLOWER
I20260811 06:22:19.506201 30006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } }
I20260811 06:22:19.506469 30006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:19.506608 30006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:19.506758 30006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:19.509449 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:19.498985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:54773 (local address 127.0.0.1:45815)
0811 06:22:19.499164 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:19.499170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:19.503840 (+ 4670us) server_negotiation.cc:407] Connection header received
0811 06:22:19.504003 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:19.504006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:19.504060 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:19.504142 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:19.504739 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:19.505431 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:19.507179 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:19.507671 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:19.507771 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:19.508814 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:19.508818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:19.508821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:19.508837 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:19.508871 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:19.508885 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:19.508958 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:19.509115 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 06:22:19.509152 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:22:19.509500 30006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } }
I20260811 06:22:19.510025 30006 leader_election.cc:302] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cb0b58daf4d42a7ba12ac81b3c53580; no voters:
I20260811 06:22:19.510556 30006 leader_election.cc:288] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:19.510689 30020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:19.511335 30020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [term 1 LEADER]: Becoming Leader. State: Replica: 2cb0b58daf4d42a7ba12ac81b3c53580, State: Running, Role: LEADER
I20260811 06:22:19.511828 30020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } }
I20260811 06:22:19.512557 30006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:19.514257 30021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } } }
I20260811 06:22:19.514672 30021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:19.515443 30022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cb0b58daf4d42a7ba12ac81b3c53580. Latest consensus state: current_term: 1 leader_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb0b58daf4d42a7ba12ac81b3c53580" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45815 } } }
I20260811 06:22:19.515725 30022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:19.516578 30035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:19.521243 30035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:19.530864 30035 catalog_manager.cc:1347] Generated new cluster ID: 27e8261672f34cf29c1fe89164f6d76c
I20260811 06:22:19.530964 30035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:19.542613 30035 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:19.544291 30035 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:19.553411 30035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cb0b58daf4d42a7ba12ac81b3c53580: Generated new TSK 0
I20260811 06:22:19.554292 30035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:19.560927 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29872
W20260811 06:22:19.562553 29993 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.164:38707
W20260811 06:22:19.563241 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 29872: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:19.563396 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 29872
W20260811 06:22:19.595420 29835 connection.cc:535] client connection to 127.0.0.1:45815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45815): Connection reset by peer (error 104)
2026-08-11T06:22:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:19 dist-test-slave-gjrz krb5kdc[30061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:19 dist-test-slave-gjrz krb5kdc[30061](info): set up 2 sockets
Aug 11 06:22:19 dist-test-slave-gjrz krb5kdc[30061](info): commencing operation
krb5kdc: starting...
W20260811 06:22:20.820330 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:20 dist-test-slave-gjrz krb5kdc[30061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:22:21.062645 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:21.376636 30093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:21.376901 30093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:21.377048 30093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:21.384042 30093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:21.384158 30093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:21.384238 30093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:21.384320 30093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:21.384378 30093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:21.393615 30093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:21.394734 30093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:21.395970 30093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:21.406705 30103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:21.406852 30105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:21.406983 30110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:21.407995 30093 server_base.cc:1034] running on GCE node
I20260811 06:22:21.408671 30093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:21.409924 30093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:21.411149 30093 hybrid_clock.cc:648] HybridClock initialized: now 1786429341411109 us; error 36 us; skew 500 ppm
I20260811 06:22:21.411477 30093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:21 dist-test-slave-gjrz krb5kdc[30061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:21.414398 30093 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:22:21.415772 30093 webserver.cc:460] Webserver started at http://127.0.0.1:40277/ using document root <none> and password file <none>
I20260811 06:22:21.416369 30093 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:21.416483 30093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:21.416751 30093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:21.419514 30093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "fad48692523b4384bf72ca025d84348b"
format_stamp: "Formatted at 2026-08-11 06:22:21 on dist-test-slave-gjrz"
server_key: "d13509a58f462b0b9cb58209692ca2d1"
server_key_iv: "ab1bc5af56be158ee20792ccdf95bfcc"
server_key_version: "encryptionkey@0"
I20260811 06:22:21.420228 30093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "fad48692523b4384bf72ca025d84348b"
format_stamp: "Formatted at 2026-08-11 06:22:21 on dist-test-slave-gjrz"
server_key: "d13509a58f462b0b9cb58209692ca2d1"
server_key_iv: "ab1bc5af56be158ee20792ccdf95bfcc"
server_key_version: "encryptionkey@0"
I20260811 06:22:21.424929 30093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:21.428048 30122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:21.429297 30093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:22:21.429459 30093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "fad48692523b4384bf72ca025d84348b"
format_stamp: "Formatted at 2026-08-11 06:22:21 on dist-test-slave-gjrz"
server_key: "d13509a58f462b0b9cb58209692ca2d1"
server_key_iv: "ab1bc5af56be158ee20792ccdf95bfcc"
server_key_version: "encryptionkey@0"
I20260811 06:22:21.429661 30093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:21.435019 30093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:21.438506 30093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:21.438735 30093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:21.455638 30093 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44115
I20260811 06:22:21.455708 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44115 every 8 connection(s)
I20260811 06:22:21.456853 30093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:21.460256 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:21.462234 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30093
I20260811 06:22:21.462419 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:21.462949 23012 external_mini_cluster.cc:1428] Setting key fb1f238fa56c0121b69fa823430688fb
I20260811 06:22:21.470160 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:21.464683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:44631 (local address 127.0.0.1:44115)
0811 06:22:21.465437 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:21.465447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:21.465493 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:22:21.467116 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:21.467138 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:21.467489 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:21.467784 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:21.468738 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:21.468744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:21.468756 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:21.468770 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:21.468828 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:21.468850 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:21.469019 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:21.469111 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 06:22:21.469293 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":345,"threads_started":1}
I20260811 06:22:21.471654 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b: Bootstrap starting.
I20260811 06:22:21.473368 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:21.474828 30228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:21.476023 30228 log.cc:824] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:21.478720 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b: No bootstrap required, opened a new log
I20260811 06:22:21.480809 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:21.478297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:38153 (local address 127.0.0.1:44115)
0811 06:22:21.478462 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:21.478468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:21.478984 (+ 516us) server_negotiation.cc:407] Connection header received
0811 06:22:21.479097 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:21.479100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:21.479149 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:21.479232 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:21.480171 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:21.480176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:21.480179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:21.480194 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:21.480231 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:21.480246 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:21.480350 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:21.480486 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 06:22:21.480533 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:22:21.486269 30228 raft_consensus.cc:348] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } }
I20260811 06:22:21.486634 30228 raft_consensus.cc:374] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:21.486738 30228 raft_consensus.cc:729] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad48692523b4384bf72ca025d84348b, State: Initialized, Role: FOLLOWER
I20260811 06:22:21.487353 30228 consensus_queue.cc:260] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } }
I20260811 06:22:21.487579 30228 raft_consensus.cc:388] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:21.487713 30228 raft_consensus.cc:482] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:21.487854 30228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:21.490008 30228 raft_consensus.cc:504] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } }
I20260811 06:22:21.490500 30228 leader_election.cc:302] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fad48692523b4384bf72ca025d84348b; no voters:
I20260811 06:22:21.493559 30228 leader_election.cc:288] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:21.493696 30248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:21.495744 30228 sys_catalog.cc:564] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:21.496737 30248 raft_consensus.cc:686] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [term 1 LEADER]: Becoming Leader. State: Replica: fad48692523b4384bf72ca025d84348b, State: Running, Role: LEADER
I20260811 06:22:21.497300 30248 consensus_queue.cc:237] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } }
I20260811 06:22:21.500114 30252 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fad48692523b4384bf72ca025d84348b. Latest consensus state: current_term: 1 leader_uuid: "fad48692523b4384bf72ca025d84348b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } } }
I20260811 06:22:21.500587 30252 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:21.500440 30251 sys_catalog.cc:455] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fad48692523b4384bf72ca025d84348b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fad48692523b4384bf72ca025d84348b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44115 } } }
I20260811 06:22:21.500815 30251 sys_catalog.cc:458] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:21.502452 30262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:21.506184 30262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:21.513429 30262 catalog_manager.cc:1347] Generated new cluster ID: 2e1410f4d9fe4c7f873cf602fb49c92c
I20260811 06:22:21.513521 30262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:21.539182 30262 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:21.539630 30262 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:21.547967 30262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fad48692523b4384bf72ca025d84348b: Generated new TSK 0
I20260811 06:22:21.548642 30262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:21.557247 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30093
W20260811 06:22:21.558276 30218 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.164:44631
W20260811 06:22:21.558877 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30093: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:21.559026 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30093
2026-08-11T06:22:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1986 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:21 dist-test-slave-gjrz krb5kdc[30278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:21 dist-test-slave-gjrz krb5kdc[30278](info): set up 2 sockets
Aug 11 06:22:21 dist-test-slave-gjrz krb5kdc[30278](info): commencing operation
krb5kdc: starting...
W20260811 06:22:22.786581 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:22 dist-test-slave-gjrz krb5kdc[30278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:23.028661 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43801
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:23.352629 30304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:23.352941 30304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:23.353076 30304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:23.359957 30304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:23.360064 30304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:23.360141 30304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:23.360224 30304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:23.360285 30304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:23.369589 30304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:43801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:23.370716 30304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:23.371960 30304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:23.383072 30314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:23.383144 30318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:23.383152 30316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:23.383878 30304 server_base.cc:1034] running on GCE node
I20260811 06:22:23.384567 30304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:23.385855 30304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:23.387055 30304 hybrid_clock.cc:648] HybridClock initialized: now 1786429343387012 us; error 31 us; skew 500 ppm
I20260811 06:22:23.387399 30304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:23 dist-test-slave-gjrz krb5kdc[30278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429343, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:23.390214 30304 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:23.391562 30304 webserver.cc:460] Webserver started at http://127.22.121.62:33039/ using document root <none> and password file <none>
I20260811 06:22:23.392170 30304 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:23.392287 30304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:23.392578 30304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:23.395399 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "5302cb4124244469831eae97a1fac06f"
format_stamp: "Formatted at 2026-08-11 06:22:23 on dist-test-slave-gjrz"
server_key: "939fca296120cf70b8b9fcb50d1e2ab1"
server_key_iv: "452b7de2039f803d8b049c19c62478a6"
server_key_version: "encryptionkey@0"
I20260811 06:22:23.396150 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "5302cb4124244469831eae97a1fac06f"
format_stamp: "Formatted at 2026-08-11 06:22:23 on dist-test-slave-gjrz"
server_key: "939fca296120cf70b8b9fcb50d1e2ab1"
server_key_iv: "452b7de2039f803d8b049c19c62478a6"
server_key_version: "encryptionkey@0"
I20260811 06:22:23.400915 30304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:23.404240 30331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:23.405412 30304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:22:23.405575 30304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "5302cb4124244469831eae97a1fac06f"
format_stamp: "Formatted at 2026-08-11 06:22:23 on dist-test-slave-gjrz"
server_key: "939fca296120cf70b8b9fcb50d1e2ab1"
server_key_iv: "452b7de2039f803d8b049c19c62478a6"
server_key_version: "encryptionkey@0"
I20260811 06:22:23.405754 30304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:23.438514 30304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:23.442435 30304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:23.442682 30304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:23.459483 30304 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:43801
I20260811 06:22:23.459502 30425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:43801 every 8 connection(s)
I20260811 06:22:23.460709 30304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:23.464244 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:23.464408 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30304
I20260811 06:22:23.464582 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:23.465019 23012 external_mini_cluster.cc:1428] Setting key b9b5e0034b0ae55a9293d69f2734009b
I20260811 06:22:23.477619 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f: Bootstrap starting.
Aug 11 06:22:23 dist-test-slave-gjrz krb5kdc[30278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:23.480882 30427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:23.481894 30427 log.cc:824] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:23.484938 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f: No bootstrap required, opened a new log
I20260811 06:22:23.487002 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:23.466587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50027 (local address 127.22.121.62:43801)
0811 06:22:23.467396 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:23.467407 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:23.467444 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:22:23.468885 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:23.468904 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:23.469267 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:23.469583 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:23.471243 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:23.472502 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:23.474269 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:23.474945 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:23.475079 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:23.479865 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:23.479872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:23.479885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:23.479922 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:23.483043 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:23.483907 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:23.483913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:23.483919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:23.484074 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:23.484529 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:23.484532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:23.484534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:23.484983 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:23.485321 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:23.485781 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 06:22:23.486070 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":338,"threads_started":1}
I20260811 06:22:23.490443 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:23.493381 30427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } }
I20260811 06:22:23.493813 30427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:23.493950 30427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5302cb4124244469831eae97a1fac06f, State: Initialized, Role: FOLLOWER
I20260811 06:22:23.494594 30427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } }
I20260811 06:22:23.494870 30427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:23.494995 30427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:23.495158 30427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:23.498155 30427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } }
I20260811 06:22:23.498736 30427 leader_election.cc:302] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5302cb4124244469831eae97a1fac06f; no voters:
I20260811 06:22:23.503551 30427 leader_election.cc:288] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:23.503607 30450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:23.504834 30450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [term 1 LEADER]: Becoming Leader. State: Replica: 5302cb4124244469831eae97a1fac06f, State: Running, Role: LEADER
I20260811 06:22:23.505381 30450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } }
I20260811 06:22:23.505641 30427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:23.507830 30451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5302cb4124244469831eae97a1fac06f. Latest consensus state: current_term: 1 leader_uuid: "5302cb4124244469831eae97a1fac06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } } }
I20260811 06:22:23.508108 30451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:23.509843 30457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:23.509711 30452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5302cb4124244469831eae97a1fac06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5302cb4124244469831eae97a1fac06f" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 43801 } } }
I20260811 06:22:23.510725 30452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:23.516516 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:23.498662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51349 (local address 127.22.121.62:43801)
0811 06:22:23.499135 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:23.499168 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:23.499904 (+ 736us) server_negotiation.cc:407] Connection header received
0811 06:22:23.500041 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:23.500047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:23.500133 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:23.500231 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:23.502269 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:23.502982 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:23.504662 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:23.505342 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:23.505477 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:23.508478 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:23.508486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:23.508491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:23.508528 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:23.512352 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:23.513331 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:23.513337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:23.513341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:23.513484 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:23.514047 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:23.514054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:23.514058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:23.514482 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:23.514750 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:23.515946 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 06:22:23.516152 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260811 06:22:23.521970 30457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:23.530464 30457 catalog_manager.cc:1347] Generated new cluster ID: 12fc0ba89c6e40b9b0a265773463aa27
I20260811 06:22:23.530570 30457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:23.541049 30457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:23.542194 30457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:23.552069 30457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5302cb4124244469831eae97a1fac06f: Generated new TSK 0
I20260811 06:22:23.552733 30457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:23.565114 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30304
I20260811 06:22:23.566568 30416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:23.653697 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30304
W20260811 06:22:23.675410 30276 connection.cc:535] client connection to 127.22.121.62:43801 recv error: Network error: failed to read from TLS socket (remote: 127.22.121.62:43801): Connection reset by peer (error 104)
2026-08-11T06:22:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2093 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:23 dist-test-slave-gjrz krb5kdc[30480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:23 dist-test-slave-gjrz krb5kdc[30480](info): set up 2 sockets
Aug 11 06:22:23 dist-test-slave-gjrz krb5kdc[30480](info): commencing operation
krb5kdc: starting...
W20260811 06:22:24.911796 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:25 dist-test-slave-gjrz krb5kdc[30480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:25.159644 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42479
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:25.475178 30505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:25.475459 30505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:25.475585 30505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:25.482461 30505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:25.482565 30505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:25.482645 30505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:25.482729 30505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:25.482790 30505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:25.493157 30505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:42479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:25.494691 30505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:25.496165 30505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:25.507478 30515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:25.507695 30519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:25.507925 30516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:25.508517 30505 server_base.cc:1034] running on GCE node
I20260811 06:22:25.509213 30505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:25.510416 30505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:25.511607 30505 hybrid_clock.cc:648] HybridClock initialized: now 1786429345511567 us; error 27 us; skew 500 ppm
I20260811 06:22:25.511917 30505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:25 dist-test-slave-gjrz krb5kdc[30480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429345, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:25.514719 30505 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:25.516048 30505 webserver.cc:460] Webserver started at http://127.22.121.62:36353/ using document root <none> and password file <none>
I20260811 06:22:25.516654 30505 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:25.516767 30505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:25.517035 30505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:25.519927 30505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "ac60e7126a8848af8c0ffb058ed39486"
format_stamp: "Formatted at 2026-08-11 06:22:25 on dist-test-slave-gjrz"
server_key: "09c8898fa592c201c7747b6337389a00"
server_key_iv: "d5beb4fbb07bae3e4ce6544ac6b6a260"
server_key_version: "encryptionkey@0"
I20260811 06:22:25.520658 30505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "ac60e7126a8848af8c0ffb058ed39486"
format_stamp: "Formatted at 2026-08-11 06:22:25 on dist-test-slave-gjrz"
server_key: "09c8898fa592c201c7747b6337389a00"
server_key_iv: "d5beb4fbb07bae3e4ce6544ac6b6a260"
server_key_version: "encryptionkey@0"
I20260811 06:22:25.525463 30505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:25.528659 30537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:25.529805 30505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:22:25.529954 30505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "ac60e7126a8848af8c0ffb058ed39486"
format_stamp: "Formatted at 2026-08-11 06:22:25 on dist-test-slave-gjrz"
server_key: "09c8898fa592c201c7747b6337389a00"
server_key_iv: "d5beb4fbb07bae3e4ce6544ac6b6a260"
server_key_version: "encryptionkey@0"
I20260811 06:22:25.530133 30505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:25.552685 30505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:25.556370 30505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:25.556613 30505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:25.573530 30505 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:42479
I20260811 06:22:25.573552 30638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:42479 every 8 connection(s)
I20260811 06:22:25.574718 30505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:25.578145 30639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:25.578886 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30505
I20260811 06:22:25.579039 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:25.579516 23012 external_mini_cluster.cc:1428] Setting key 23e2a3a58fb8e82bed5e51491d12b02a
I20260811 06:22:25.592012 30639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486: Bootstrap starting.
I20260811 06:22:25.593680 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:25.581220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34955 (local address 127.22.121.62:42479)
0811 06:22:25.581966 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:25.581981 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:25.582029 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:22:25.584379 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:25.584407 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:25.584878 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:25.585302 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:25.585781 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:25.587624 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:25.588912 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:25.589957 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:25.590151 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:25.591122 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:25.591133 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:25.591150 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:25.591831 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:25.591922 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:25.591951 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:25.592125 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:25.592317 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:22:25.592475 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":301,"threads_started":1}
I20260811 06:22:25.595359 30639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:25.596478 30639 log.cc:824] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:25.597074 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:25.598887 30639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486: No bootstrap required, opened a new log
I20260811 06:22:25.607131 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:25.601181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34365 (local address 127.22.121.62:42479)
0811 06:22:25.601342 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:25.601347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:25.601652 (+ 305us) server_negotiation.cc:407] Connection header received
0811 06:22:25.601767 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:25.601775 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:25.601841 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:25.601957 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:25.602352 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:25.603445 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:25.604666 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:25.605316 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:25.605458 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:25.606402 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:25.606412 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:25.606417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:25.606439 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:25.606502 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:25.606524 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:25.606629 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:25.606774 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 06:22:25.606827 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:22:25.607694 30639 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } }
I20260811 06:22:25.608026 30639 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:25.608134 30639 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac60e7126a8848af8c0ffb058ed39486, State: Initialized, Role: FOLLOWER
I20260811 06:22:25.608704 30639 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } }
I20260811 06:22:25.608909 30639 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:25.609004 30639 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:25.609127 30639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:25.611068 30639 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } }
I20260811 06:22:25.611550 30639 leader_election.cc:302] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac60e7126a8848af8c0ffb058ed39486; no voters:
I20260811 06:22:25.612013 30639 leader_election.cc:288] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:25.612128 30651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:25.612706 30651 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [term 1 LEADER]: Becoming Leader. State: Replica: ac60e7126a8848af8c0ffb058ed39486, State: Running, Role: LEADER
I20260811 06:22:25.613209 30651 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } }
I20260811 06:22:25.613821 30639 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:25.615899 30652 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac60e7126a8848af8c0ffb058ed39486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } } }
I20260811 06:22:25.615948 30653 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac60e7126a8848af8c0ffb058ed39486. Latest consensus state: current_term: 1 leader_uuid: "ac60e7126a8848af8c0ffb058ed39486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac60e7126a8848af8c0ffb058ed39486" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42479 } } }
I20260811 06:22:25.616299 30652 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:25.616302 30653 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:25.617874 30662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:25.622256 30662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:25.629832 30662 catalog_manager.cc:1347] Generated new cluster ID: d07cb0bb51644cd49b0b5b9816835630
I20260811 06:22:25.629931 30662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:25.644135 30662 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:25.645339 30662 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:25.651233 30662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac60e7126a8848af8c0ffb058ed39486: Generated new TSK 0
I20260811 06:22:25.651948 30662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:25.665524 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30505
W20260811 06:22:25.666513 30630 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34955
W20260811 06:22:25.667164 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30505: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:25.667338 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30505
W20260811 06:22:25.691308 30477 connection.cc:535] client connection to 127.22.121.62:42479 recv error: Network error: failed to read from TLS socket (remote: 127.22.121.62:42479): Connection reset by peer (error 104)
2026-08-11T06:22:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2015 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:25 dist-test-slave-gjrz krb5kdc[30689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:25 dist-test-slave-gjrz krb5kdc[30689](info): set up 2 sockets
Aug 11 06:22:25 dist-test-slave-gjrz krb5kdc[30689](info): commencing operation
krb5kdc: starting...
W20260811 06:22:26.898370 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:27 dist-test-slave-gjrz krb5kdc[30689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:27.143666 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42777
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:27.460983 30722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:27.461236 30722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:27.461365 30722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:27.468266 30722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:27.468367 30722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:27.468452 30722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:27.468525 30722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:27.468582 30722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:27.478477 30722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:42777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:27.479645 30722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:27.480870 30722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:27.491863 30735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:27.491918 30743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:27.492868 30737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:27.492832 30722 server_base.cc:1034] running on GCE node
I20260811 06:22:27.493808 30722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:27.495291 30722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:27.496510 30722 hybrid_clock.cc:648] HybridClock initialized: now 1786429347496455 us; error 39 us; skew 500 ppm
I20260811 06:22:27.496920 30722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:27 dist-test-slave-gjrz krb5kdc[30689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429347, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:27.500497 30722 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:27.501838 30722 webserver.cc:460] Webserver started at http://127.22.121.62:43389/ using document root <none> and password file <none>
I20260811 06:22:27.502468 30722 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:27.502591 30722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:27.502875 30722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:27.505738 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "c54bf770f6aa46b5a9a1a6eed84012b7"
format_stamp: "Formatted at 2026-08-11 06:22:27 on dist-test-slave-gjrz"
server_key: "e2970e4f5a6546d6115fb45662412685"
server_key_iv: "0bdf4877dd1e028cd0eaedf6db90670c"
server_key_version: "encryptionkey@0"
I20260811 06:22:27.506489 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "c54bf770f6aa46b5a9a1a6eed84012b7"
format_stamp: "Formatted at 2026-08-11 06:22:27 on dist-test-slave-gjrz"
server_key: "e2970e4f5a6546d6115fb45662412685"
server_key_iv: "0bdf4877dd1e028cd0eaedf6db90670c"
server_key_version: "encryptionkey@0"
I20260811 06:22:27.511387 30722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:22:27.514573 30752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:27.515815 30722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:22:27.515969 30722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "c54bf770f6aa46b5a9a1a6eed84012b7"
format_stamp: "Formatted at 2026-08-11 06:22:27 on dist-test-slave-gjrz"
server_key: "e2970e4f5a6546d6115fb45662412685"
server_key_iv: "0bdf4877dd1e028cd0eaedf6db90670c"
server_key_version: "encryptionkey@0"
I20260811 06:22:27.516139 30722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:27.521334 30722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:27.525286 30722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:27.525539 30722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:27.541461 30722 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:42777
I20260811 06:22:27.541497 30841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:42777 every 8 connection(s)
I20260811 06:22:27.542680 30722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:27.544164 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30722
I20260811 06:22:27.544309 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:27.544744 23012 external_mini_cluster.cc:1428] Setting key c8bd2465704f6cfc3b759e7c486b0caf
I20260811 06:22:27.546468 30844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:27.552906 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:27.546411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59325 (local address 127.22.121.62:42777)
0811 06:22:27.547221 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:27.547236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:27.547302 (+ 66us) server_negotiation.cc:407] Connection header received
0811 06:22:27.549305 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:27.549331 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:27.549742 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:27.550157 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:27.551088 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:27.551107 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:27.551123 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:27.551141 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:27.551237 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:27.551291 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:27.551491 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:27.551622 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 06:22:27.551798 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":315,"threads_started":1}
I20260811 06:22:27.556193 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:27.559934 30844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7: Bootstrap starting.
I20260811 06:22:27.563238 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:27.561130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33673 (local address 127.22.121.62:42777)
0811 06:22:27.561284 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:27.561290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:27.561666 (+ 376us) server_negotiation.cc:407] Connection header received
0811 06:22:27.561777 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:27.561782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:27.561840 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:27.561952 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:27.562694 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:27.562699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:27.562703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:27.562720 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:27.562760 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:27.562777 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:27.562884 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:27.562925 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 06:22:27.562972 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:22:27.563284 30844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:27.564428 30844 log.cc:824] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:27.566872 30844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7: No bootstrap required, opened a new log
I20260811 06:22:27.573668 30844 raft_consensus.cc:348] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } }
I20260811 06:22:27.573956 30844 raft_consensus.cc:374] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:27.574033 30844 raft_consensus.cc:729] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54bf770f6aa46b5a9a1a6eed84012b7, State: Initialized, Role: FOLLOWER
I20260811 06:22:27.574471 30844 consensus_queue.cc:260] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } }
I20260811 06:22:27.574661 30844 raft_consensus.cc:388] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:27.574755 30844 raft_consensus.cc:482] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:27.574874 30844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:27.577401 30844 raft_consensus.cc:504] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } }
I20260811 06:22:27.577811 30844 leader_election.cc:302] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c54bf770f6aa46b5a9a1a6eed84012b7; no voters:
I20260811 06:22:27.578305 30844 leader_election.cc:288] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:27.578418 30857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:27.579509 30857 raft_consensus.cc:686] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [term 1 LEADER]: Becoming Leader. State: Replica: c54bf770f6aa46b5a9a1a6eed84012b7, State: Running, Role: LEADER
I20260811 06:22:27.579988 30844 sys_catalog.cc:564] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:27.579950 30857 consensus_queue.cc:237] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } }
I20260811 06:22:27.582726 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c54bf770f6aa46b5a9a1a6eed84012b7. Latest consensus state: current_term: 1 leader_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } } }
I20260811 06:22:27.583151 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:27.583478 30858 sys_catalog.cc:455] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54bf770f6aa46b5a9a1a6eed84012b7" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42777 } } }
I20260811 06:22:27.583776 30858 sys_catalog.cc:458] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:27.586457 30871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:27.590162 30871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:27.597659 30871 catalog_manager.cc:1347] Generated new cluster ID: 477c66dad174442cae2911cbbae67dee
I20260811 06:22:27.597756 30871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:27.615661 30871 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:27.616130 30871 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:27.632043 30871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c54bf770f6aa46b5a9a1a6eed84012b7: Generated new TSK 0
I20260811 06:22:27.632920 30871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:27.640161 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30722
W20260811 06:22:27.641162 30835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59325
W20260811 06:22:27.641781 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:27.641906 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30722
2026-08-11T06:22:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1974 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:27 dist-test-slave-gjrz krb5kdc[30896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:27 dist-test-slave-gjrz krb5kdc[30896](info): set up 2 sockets
Aug 11 06:22:27 dist-test-slave-gjrz krb5kdc[30896](info): commencing operation
krb5kdc: starting...
W20260811 06:22:28.882525 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:29.121622 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45307
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:45307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:29.435278 30918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:29.435563 30918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:29.435706 30918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:29.442524 30918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:29.442632 30918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:29.442708 30918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:29.442788 30918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:29.442847 30918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:29.452112 30918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:45307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:45307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:29.453272 30918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:29.454527 30918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:29.466008 30934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:29.466066 30931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:29.466044 30938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:29.466892 30918 server_base.cc:1034] running on GCE node
I20260811 06:22:29.467648 30918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:29.468938 30918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:29.470141 30918 hybrid_clock.cc:648] HybridClock initialized: now 1786429349470093 us; error 38 us; skew 500 ppm
I20260811 06:22:29.470461 30918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429349, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:29.473361 30918 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:29.474682 30918 webserver.cc:460] Webserver started at http://127.22.121.62:33513/ using document root <none> and password file <none>
I20260811 06:22:29.475302 30918 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:29.475415 30918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:29.475677 30918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:29.478437 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "0e67132087f94a268e0b094fe961892d"
format_stamp: "Formatted at 2026-08-11 06:22:29 on dist-test-slave-gjrz"
server_key: "ae2c4937140b5862ec9312147aa3c6ca"
server_key_iv: "ddf46a20ecc995390db4457d789665ac"
server_key_version: "encryptionkey@0"
I20260811 06:22:29.479135 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "0e67132087f94a268e0b094fe961892d"
format_stamp: "Formatted at 2026-08-11 06:22:29 on dist-test-slave-gjrz"
server_key: "ae2c4937140b5862ec9312147aa3c6ca"
server_key_iv: "ddf46a20ecc995390db4457d789665ac"
server_key_version: "encryptionkey@0"
I20260811 06:22:29.484000 30918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 06:22:29.487147 30947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:29.488363 30918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:22:29.488495 30918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "0e67132087f94a268e0b094fe961892d"
format_stamp: "Formatted at 2026-08-11 06:22:29 on dist-test-slave-gjrz"
server_key: "ae2c4937140b5862ec9312147aa3c6ca"
server_key_iv: "ddf46a20ecc995390db4457d789665ac"
server_key_version: "encryptionkey@0"
I20260811 06:22:29.488659 30918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:29.503908 30918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:29.507464 30918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:29.507686 30918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:29.524037 30918 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:45307
I20260811 06:22:29.524066 31030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:45307 every 8 connection(s)
I20260811 06:22:29.525228 30918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:29.528640 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:29.531015 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 30918
I20260811 06:22:29.531172 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:29.531638 23012 external_mini_cluster.cc:1428] Setting key 8406631d3e217248c6b9383e5089ece0
I20260811 06:22:29.540493 31033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d: Bootstrap starting.
I20260811 06:22:29.543752 31033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:29.544667 31033 log.cc:824] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:29.547461 31033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d: No bootstrap required, opened a new log
I20260811 06:22:29.553673 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:29.533318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48067 (local address 127.22.121.62:45307)
0811 06:22:29.534085 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:29.534099 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:29.534147 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:22:29.536355 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:29.536383 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:29.536861 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:29.537268 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:29.538526 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:29.540423 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:29.541677 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:29.542708 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:29.542878 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:29.546808 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:29.546815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:29.546829 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:29.546861 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:29.549776 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:29.550545 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:29.550551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:29.550556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:29.550730 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:29.551108 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:29.551111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:29.551113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:29.551676 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:29.551890 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:29.552327 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 06:22:29.552653 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":324,"threads_started":1}
I20260811 06:22:29.556058 31033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } }
I20260811 06:22:29.556414 31033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:29.556510 31033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e67132087f94a268e0b094fe961892d, State: Initialized, Role: FOLLOWER
I20260811 06:22:29.557017 31033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } }
I20260811 06:22:29.557248 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:29.557281 31033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:29.557415 31033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:29.557551 31033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:29.559544 31033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } }
I20260811 06:22:29.560034 31033 leader_election.cc:302] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e67132087f94a268e0b094fe961892d; no voters:
I20260811 06:22:29.560621 31033 leader_election.cc:288] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:29.560732 31048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:29.561623 31048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [term 1 LEADER]: Becoming Leader. State: Replica: 0e67132087f94a268e0b094fe961892d, State: Running, Role: LEADER
I20260811 06:22:29.562252 31048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } }
I20260811 06:22:29.564080 31033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:29.565308 31053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e67132087f94a268e0b094fe961892d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } } }
I20260811 06:22:29.565703 31053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:29.565363 31050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e67132087f94a268e0b094fe961892d. Latest consensus state: current_term: 1 leader_uuid: "0e67132087f94a268e0b094fe961892d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e67132087f94a268e0b094fe961892d" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 45307 } } }
I20260811 06:22:29.566118 31050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:29.571388 31059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:29.576190 31059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:29.583451 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:29.561341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49231 (local address 127.22.121.62:45307)
0811 06:22:29.561486 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:29.561492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:29.562593 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 06:22:29.562830 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:29.562836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:29.562898 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:29.562973 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:29.566564 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:29.568703 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:29.571430 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:29.572167 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:29.572314 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:29.575489 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:29.575497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:29.575502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:29.575542 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:29.579470 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:29.580313 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:29.580319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:29.580324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:29.580466 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:29.580908 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:29.580913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:29.580916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:29.581375 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:29.582708 (+ 1333us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:29.582863 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 06:22:29.583078 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:22:29.591737 31059 catalog_manager.cc:1347] Generated new cluster ID: c22830f829d347bfabac6fa55c2833b0
I20260811 06:22:29.591825 31059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:29.616415 31059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:29.617720 31059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:29.629413 31059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e67132087f94a268e0b094fe961892d: Generated new TSK 0
I20260811 06:22:29.630125 31059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:29.638744 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 30918
I20260811 06:22:29.639914 31023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:29.729984 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 30918
2026-08-11T06:22:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[31093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[31093](info): set up 2 sockets
Aug 11 06:22:29 dist-test-slave-gjrz krb5kdc[31093](info): commencing operation
krb5kdc: starting...
W20260811 06:22:30.972978 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:31 dist-test-slave-gjrz krb5kdc[31093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:31.221799 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:44351
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:31.540244 31113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:31.540505 31113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:31.540640 31113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:31.547529 31113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:31.547634 31113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:31.547711 31113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:31.547794 31113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:31.547854 31113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:31.556999 31113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:31.558125 31113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:31.559464 31113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:31.570652 31121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:31.570573 31128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:31.570623 31125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:31.571467 31113 server_base.cc:1034] running on GCE node
I20260811 06:22:31.572187 31113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:31.573439 31113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:31.574628 31113 hybrid_clock.cc:648] HybridClock initialized: now 1786429351574594 us; error 31 us; skew 500 ppm
I20260811 06:22:31.574944 31113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:31 dist-test-slave-gjrz krb5kdc[31093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429351, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:31.581802 31113 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:31.583190 31113 webserver.cc:460] Webserver started at http://127.22.121.62:34671/ using document root <none> and password file <none>
I20260811 06:22:31.583822 31113 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:31.583937 31113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:31.584213 31113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:31.586979 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "aa3b297760394779b2568983ee352a11"
format_stamp: "Formatted at 2026-08-11 06:22:31 on dist-test-slave-gjrz"
server_key: "b29110335dbda358c23101fd0799c68b"
server_key_iv: "0b5c99a59c9ddd0fe9b24633ffa67ec1"
server_key_version: "encryptionkey@0"
I20260811 06:22:31.587699 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "aa3b297760394779b2568983ee352a11"
format_stamp: "Formatted at 2026-08-11 06:22:31 on dist-test-slave-gjrz"
server_key: "b29110335dbda358c23101fd0799c68b"
server_key_iv: "0b5c99a59c9ddd0fe9b24633ffa67ec1"
server_key_version: "encryptionkey@0"
I20260811 06:22:31.592321 31113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:22:31.595448 31137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:31.596661 31113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:22:31.596815 31113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "aa3b297760394779b2568983ee352a11"
format_stamp: "Formatted at 2026-08-11 06:22:31 on dist-test-slave-gjrz"
server_key: "b29110335dbda358c23101fd0799c68b"
server_key_iv: "0b5c99a59c9ddd0fe9b24633ffa67ec1"
server_key_version: "encryptionkey@0"
I20260811 06:22:31.596978 31113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:31.635306 31113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:31.639175 31113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:31.639438 31113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:31.656045 31113 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:44351
I20260811 06:22:31.656060 31214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:44351 every 8 connection(s)
I20260811 06:22:31.657224 31113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:31.660694 31217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:31.661834 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31113
I20260811 06:22:31.662029 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:31.662601 23012 external_mini_cluster.cc:1428] Setting key 98bb3a1977978972e81b2bd72db3eca1
I20260811 06:22:31.673753 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11: Bootstrap starting.
I20260811 06:22:31.674799 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:31.664291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60577 (local address 127.22.121.62:44351)
0811 06:22:31.665033 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:31.665043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:31.665080 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:22:31.666668 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:31.666692 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:31.667043 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:31.667393 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:31.667928 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:31.669128 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:31.670911 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:31.671631 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:31.671761 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:31.672880 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:31.672887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:31.672900 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:31.673384 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:31.673453 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:31.673476 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:31.673555 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:31.673760 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:22:31.673963 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":334,"threads_started":1}
I20260811 06:22:31.676770 31217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:31.677902 31217 log.cc:824] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:31.678103 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:31.680415 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11: No bootstrap required, opened a new log
I20260811 06:22:31.687776 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:31.682334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32977 (local address 127.22.121.62:44351)
0811 06:22:31.682476 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:31.682481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:31.682876 (+ 395us) server_negotiation.cc:407] Connection header received
0811 06:22:31.683031 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:31.683035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:31.683099 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:31.683178 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:31.683667 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:31.684346 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:31.685597 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:31.686039 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:31.686142 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:31.687104 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:31.687108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:31.687111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:31.687126 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:31.687179 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:31.687194 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:31.687289 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:31.687431 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 06:22:31.687511 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 06:22:31.688252 31217 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } }
I20260811 06:22:31.688661 31217 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:31.688778 31217 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3b297760394779b2568983ee352a11, State: Initialized, Role: FOLLOWER
I20260811 06:22:31.689365 31217 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } }
I20260811 06:22:31.689579 31217 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:31.689672 31217 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:31.689790 31217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:31.691804 31217 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } }
I20260811 06:22:31.692209 31217 leader_election.cc:302] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3b297760394779b2568983ee352a11; no voters:
I20260811 06:22:31.692651 31217 leader_election.cc:288] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:31.692770 31237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:31.693416 31237 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [term 1 LEADER]: Becoming Leader. State: Replica: aa3b297760394779b2568983ee352a11, State: Running, Role: LEADER
I20260811 06:22:31.693895 31237 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } }
I20260811 06:22:31.694543 31217 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:31.696545 31240 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3b297760394779b2568983ee352a11. Latest consensus state: current_term: 1 leader_uuid: "aa3b297760394779b2568983ee352a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } } }
I20260811 06:22:31.696569 31239 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3b297760394779b2568983ee352a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3b297760394779b2568983ee352a11" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 44351 } } }
I20260811 06:22:31.697011 31240 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:31.697021 31239 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:31.698621 31247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:31.703855 31247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:31.712265 31247 catalog_manager.cc:1347] Generated new cluster ID: 21a23b5fd82c46df808cd1d042570135
I20260811 06:22:31.712352 31247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:31.726842 31247 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:31.728585 31247 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:31.737253 31247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3b297760394779b2568983ee352a11: Generated new TSK 0
I20260811 06:22:31.737944 31247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:31.753068 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31113
W20260811 06:22:31.754070 31208 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60577
W20260811 06:22:31.754732 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31113: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:31.754880 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31113
2026-08-11T06:22:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:31 dist-test-slave-gjrz krb5kdc[31275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:31 dist-test-slave-gjrz krb5kdc[31275](info): set up 2 sockets
Aug 11 06:22:31 dist-test-slave-gjrz krb5kdc[31275](info): commencing operation
krb5kdc: starting...
W20260811 06:22:32.996328 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:33 dist-test-slave-gjrz krb5kdc[31275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:33.240527 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42859
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:42859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:33.560693 31295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:33.560942 31295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:33.561038 31295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:33.567935 31295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:33.568027 31295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:33.568079 31295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:33.568137 31295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:33.568181 31295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:33.577370 31295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:42859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:42859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:33.578430 31295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:33.579666 31295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:33.590523 31302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:33.590559 31306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:33.590595 31309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:33.591250 31295 server_base.cc:1034] running on GCE node
I20260811 06:22:33.591969 31295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:33.593178 31295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:33.594373 31295 hybrid_clock.cc:648] HybridClock initialized: now 1786429353594325 us; error 44 us; skew 500 ppm
I20260811 06:22:33.594684 31295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:33 dist-test-slave-gjrz krb5kdc[31275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429353, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:33.597558 31295 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:33.598878 31295 webserver.cc:460] Webserver started at http://127.22.121.62:41667/ using document root <none> and password file <none>
I20260811 06:22:33.599524 31295 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:33.599624 31295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:33.599860 31295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:33.602607 31295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "06731e55f9d44634a3791aef17c942a0"
format_stamp: "Formatted at 2026-08-11 06:22:33 on dist-test-slave-gjrz"
server_key: "c55bd8b07b7a5d6d05d7ca58585b3ac4"
server_key_iv: "df37f5c80b9ec3860d810c644013c515"
server_key_version: "encryptionkey@0"
I20260811 06:22:33.603319 31295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "06731e55f9d44634a3791aef17c942a0"
format_stamp: "Formatted at 2026-08-11 06:22:33 on dist-test-slave-gjrz"
server_key: "c55bd8b07b7a5d6d05d7ca58585b3ac4"
server_key_iv: "df37f5c80b9ec3860d810c644013c515"
server_key_version: "encryptionkey@0"
I20260811 06:22:33.607988 31295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 06:22:33.611088 31319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:33.612267 31295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:22:33.612402 31295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "06731e55f9d44634a3791aef17c942a0"
format_stamp: "Formatted at 2026-08-11 06:22:33 on dist-test-slave-gjrz"
server_key: "c55bd8b07b7a5d6d05d7ca58585b3ac4"
server_key_iv: "df37f5c80b9ec3860d810c644013c515"
server_key_version: "encryptionkey@0"
I20260811 06:22:33.612577 31295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:33.617897 31295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:33.621348 31295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:33.621608 31295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:33.638535 31295 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:42859
I20260811 06:22:33.638568 31414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:42859 every 8 connection(s)
I20260811 06:22:33.639763 31295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:33.643088 31415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:33.643194 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31295
I20260811 06:22:33.643366 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:33.643791 23012 external_mini_cluster.cc:1428] Setting key ef71f29a515077472ffde072727110ee
I20260811 06:22:33.651515 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:33.645356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46261 (local address 127.22.121.62:42859)
0811 06:22:33.646202 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:33.646220 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:33.646270 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:22:33.647952 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:33.647988 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:33.648393 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:33.648811 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:33.649730 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:33.649741 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:33.649757 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:33.649776 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:33.649840 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:33.649875 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:33.650101 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:33.650227 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:22:33.650404 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":334,"threads_started":1}
I20260811 06:22:33.654680 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:33.656410 31415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0: Bootstrap starting.
I20260811 06:22:33.659554 31415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:33.660655 31415 log.cc:824] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:33.660791 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:33.658675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56047 (local address 127.22.121.62:42859)
0811 06:22:33.658828 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:33.658833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:33.659130 (+ 297us) server_negotiation.cc:407] Connection header received
0811 06:22:33.659236 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:33.659241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:33.659333 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:33.659439 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:33.660194 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:33.660199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:33.660204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:22:33.660221 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:33.660262 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:22:33.660280 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:33.660397 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:33.660453 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 06:22:33.660502 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:22:33.663549 31415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0: No bootstrap required, opened a new log
I20260811 06:22:33.670511 31415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } }
I20260811 06:22:33.670816 31415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:33.670885 31415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06731e55f9d44634a3791aef17c942a0, State: Initialized, Role: FOLLOWER
I20260811 06:22:33.671378 31415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } }
I20260811 06:22:33.671597 31415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:33.671694 31415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:33.671803 31415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:33.673718 31415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } }
I20260811 06:22:33.674119 31415 leader_election.cc:302] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06731e55f9d44634a3791aef17c942a0; no voters:
I20260811 06:22:33.674598 31415 leader_election.cc:288] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:33.674727 31433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:33.675323 31433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [term 1 LEADER]: Becoming Leader. State: Replica: 06731e55f9d44634a3791aef17c942a0, State: Running, Role: LEADER
I20260811 06:22:33.675841 31433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } }
I20260811 06:22:33.676369 31415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:33.678579 31434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06731e55f9d44634a3791aef17c942a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } } }
I20260811 06:22:33.679051 31434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:33.678622 31435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06731e55f9d44634a3791aef17c942a0. Latest consensus state: current_term: 1 leader_uuid: "06731e55f9d44634a3791aef17c942a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06731e55f9d44634a3791aef17c942a0" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 42859 } } }
I20260811 06:22:33.679690 31435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:33.684716 31441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:33.689342 31441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:33.697741 31441 catalog_manager.cc:1347] Generated new cluster ID: 1b2b89700be145daa175b2a4ab019186
I20260811 06:22:33.697842 31441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:33.710134 31441 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:33.710481 31441 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:33.716565 31441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06731e55f9d44634a3791aef17c942a0: Generated new TSK 0
I20260811 06:22:33.717185 31441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:33.735491 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31295
W20260811 06:22:33.736528 31403 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46261
W20260811 06:22:33.737151 23012 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31295: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:22:33.737267 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31295
2026-08-11T06:22:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1976 ms)
[----------] 18 tests from AuthTokenIssuingTest (36511 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:33 dist-test-slave-gjrz krb5kdc[31462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:33 dist-test-slave-gjrz krb5kdc[31462](info): set up 2 sockets
Aug 11 06:22:33 dist-test-slave-gjrz krb5kdc[31462](info): commencing operation
krb5kdc: starting...
W20260811 06:22:34.990528 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:35 dist-test-slave-gjrz krb5kdc[31462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:35.239719 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38991
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:35.557824 31485 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:35.558111 31485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:35.558185 31485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:35.558264 31485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:35.565145 31485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:35.565219 31485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:35.565280 31485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:35.565328 31485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:35.565387 31485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:35.574522 31485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:35.575606 31485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:35.576794 31485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:35.587604 31501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:35.587672 31503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:35.587648 31500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:35.588366 31485 server_base.cc:1034] running on GCE node
I20260811 06:22:35.589057 31485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:35.590229 31485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:35.591420 31485 hybrid_clock.cc:648] HybridClock initialized: now 1786429355591360 us; error 37 us; skew 500 ppm
I20260811 06:22:35.591713 31485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:35 dist-test-slave-gjrz krb5kdc[31462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:35.594607 31485 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:35.595969 31485 webserver.cc:460] Webserver started at http://127.22.121.62:46839/ using document root <none> and password file <none>
I20260811 06:22:35.596581 31485 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:35.596678 31485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:35.596963 31485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:35.599776 31485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "47deae91ec664255a2d425567027f3d2"
format_stamp: "Formatted at 2026-08-11 06:22:35 on dist-test-slave-gjrz"
server_key: "43e4a9d2906d070e647023530cca35a1"
server_key_iv: "d45614361f7004212f305278547522c6"
server_key_version: "encryptionkey@0"
I20260811 06:22:35.600515 31485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "47deae91ec664255a2d425567027f3d2"
format_stamp: "Formatted at 2026-08-11 06:22:35 on dist-test-slave-gjrz"
server_key: "43e4a9d2906d070e647023530cca35a1"
server_key_iv: "d45614361f7004212f305278547522c6"
server_key_version: "encryptionkey@0"
I20260811 06:22:35.605211 31485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:35.608297 31515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:35.609503 31485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:35.609654 31485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "47deae91ec664255a2d425567027f3d2"
format_stamp: "Formatted at 2026-08-11 06:22:35 on dist-test-slave-gjrz"
server_key: "43e4a9d2906d070e647023530cca35a1"
server_key_iv: "d45614361f7004212f305278547522c6"
server_key_version: "encryptionkey@0"
I20260811 06:22:35.609835 31485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:35.626904 31485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:35.630534 31485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:35.630764 31485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:35.647466 31485 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:38991
I20260811 06:22:35.647476 31604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:38991 every 8 connection(s)
I20260811 06:22:35.648634 31485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:35.649997 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31485
I20260811 06:22:35.650225 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:35.651018 23012 external_mini_cluster.cc:1428] Setting key 69ce83f8ba472d244e5a097926e01f8b
I20260811 06:22:35.652803 31605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:35.660702 31605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:35 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.121.61@KRBTEST.COM; defaulting to no policy
I20260811 06:22:35.683331 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.663958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38991 (local address 127.0.0.1:38544)
0811 06:22:35.664745 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.664789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:35.667027 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:35.667835 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:35.667842 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:35.668598 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:35.669007 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:35.669030 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:35.670508 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:35.670512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:35.672126 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:35.672141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:35.673007 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:35.673015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:35.673175 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:35.673769 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:35.673796 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:35.676125 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:35.679808 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:35.679817 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:35.679826 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:35.680196 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:35.680645 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:35.680648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:35.680651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:35.680789 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:35.681646 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:35.681651 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:35.681822 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 06:22:35.682016 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 06:22:35.682337 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":335,"threads_started":1}
I20260811 06:22:35.683351 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.663959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.22.121.62:38991)
0811 06:22:35.665303 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:35.665312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:35.665352 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:22:35.667388 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:35.667393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:35.667609 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:35.667762 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:35.669207 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:35.670355 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:35.672249 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:35.672765 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:35.672890 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:35.676356 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:35.676366 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:35.676379 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:35.676417 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:35.679692 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:35.680333 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:35.680341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:35.680345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:35.680537 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:35.680891 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:35.680895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:35.680899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:35.681512 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:35.681726 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:35.682216 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 06:22:35.682461 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":971,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.61@KRBTEST.COM" created.
I20260811 06:22:35.685842 31605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:35.686756 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.686420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49402)
0811 06:22:35.686563 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.686669 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 06:22:35.687039 31518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
I20260811 06:22:35.688441 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.688146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49414)
0811 06:22:35.688263 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.688325 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:22:35.688625 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:39071: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
I20260811 06:22:35.714347 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:39071
--webserver_interface=127.22.121.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:22:35.737887 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } attempt: 1
I20260811 06:22:35.738952 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.738543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49420)
0811 06:22:35.738721 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.738823 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:22:35.739892 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.739576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49434)
0811 06:22:35.739706 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.739772 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 06:22:35.740056 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:39071: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
I20260811 06:22:35.808249 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } attempt: 2
I20260811 06:22:35.809357 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.808928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49444)
0811 06:22:35.809140 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.809228 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:22:35.810297 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.809966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49456)
0811 06:22:35.810090 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.810169 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:22:35.810488 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:39071: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
I20260811 06:22:35.901717 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } attempt: 3
I20260811 06:22:35.902871 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.902454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49458)
0811 06:22:35.902647 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.902740 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 06:22:35.903955 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:35.903576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49468)
0811 06:22:35.903756 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:35.903825 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 06:22:35.904143 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:39071: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
W20260811 06:22:36.030977 31620 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:36.031280 31620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:36.031368 31620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:36.031447 31620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:36.038630 31620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:36.038722 31620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:36.038811 31620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:36.038878 31620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:36.038961 31620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:36.047395 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } attempt: 4
I20260811 06:22:36.048560 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.047978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49478)
0811 06:22:36.048287 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.048397 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 06:22:36.048573 31620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.121.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:36.049603 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.049206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49486)
0811 06:22:36.049349 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.049447 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:22:36.049791 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:39071: Network error: Client connection negotiation failed: client connection to 127.22.121.61:39071: connect: Connection refused (error 111)
I20260811 06:22:36.050089 31620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:36.051236 31620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:36.061813 31625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:36.061847 31628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:36.061872 31626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:36.062619 31620 server_base.cc:1034] running on GCE node
I20260811 06:22:36.063297 31620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:36.064505 31620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:36.065716 31620 hybrid_clock.cc:648] HybridClock initialized: now 1786429356065660 us; error 25 us; skew 500 ppm
I20260811 06:22:36.066004 31620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:36.069015 31620 init.cc:377] Logged in from keytab as kudu/127.22.121.61@KRBTEST.COM (short username kudu)
I20260811 06:22:36.070271 31620 webserver.cc:460] Webserver started at http://127.22.121.61:37111/ using document root <none> and password file <none>
I20260811 06:22:36.070881 31620 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:36.070981 31620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:36.071441 31620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:36.074910 31620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data/instance:
uuid: "f8c81577f5cf47fca482c4a4b38b202e"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6868a0f3b0bd37a349d68a49c6015f62"
server_key_iv: "df7caf3584cc8a11a280a4150be81813"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.075690 31620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal/instance:
uuid: "f8c81577f5cf47fca482c4a4b38b202e"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6868a0f3b0bd37a349d68a49c6015f62"
server_key_iv: "df7caf3584cc8a11a280a4150be81813"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.080359 31620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 06:22:36.083510 31637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:36.084666 31620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:36.084812 31620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "f8c81577f5cf47fca482c4a4b38b202e"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6868a0f3b0bd37a349d68a49c6015f62"
server_key_iv: "df7caf3584cc8a11a280a4150be81813"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.085007 31620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:36.099848 31620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:36.103717 31620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:36.103940 31620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:36.119480 31620 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.61:39071
I20260811 06:22:36.119499 31735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.61:39071 every 8 connection(s)
I20260811 06:22:36.120659 31620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
I20260811 06:22:36.123982 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:36.127388 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31620
I20260811 06:22:36.127552 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-1/wal/instance
I20260811 06:22:36.127954 23012 external_mini_cluster.cc:1428] Setting key 42428ad99a971d8963fca063ec2b7548
I20260811 06:22:36.130039 31736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:36.148685 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.132297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.22.121.62:38991)
0811 06:22:36.132491 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.132498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.133603 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 06:22:36.135359 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.135366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.135443 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.135549 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.137061 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.137978 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.139450 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.139991 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.140095 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.143356 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.143361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.143365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.143398 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.145899 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.146550 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.146554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.146557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.146682 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.147046 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.147051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.147054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.147357 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.147486 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.148129 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.148361 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:22:36.149473 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.132733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38991 (local address 127.0.0.1:38554)
0811 06:22:36.133437 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.133479 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.135107 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.135708 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.135715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.136181 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.136880 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.136899 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.138136 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.138140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.139303 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.139318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.140207 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.140213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.140473 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.141112 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.141143 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.143144 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.146026 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.146035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.146049 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.146443 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.146782 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.146787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.146791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.146945 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.147479 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.147486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.147784 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.148021 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.148456 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":293,"threads_started":1}
I20260811 06:22:36.151139 31736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.121.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.60@KRBTEST.COM" created.
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:36.164850 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.151789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49502)
0811 06:22:36.151967 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.151986 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.152088 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.153054 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.153059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.153496 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.153681 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.153692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.155004 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.155010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.156216 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.156228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.156986 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.156991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.157107 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.157531 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.157548 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.159655 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.162472 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.162476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.162479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.162923 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.163248 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.163253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.163256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.163446 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.163970 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.163974 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.164101 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.164228 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.164491 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:22:36.164877 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.152015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.22.121.61:39071)
0811 06:22:36.152639 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.152645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.152668 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:22:36.152706 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.152711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.152855 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.152974 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.153839 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.154860 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.156331 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.156857 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.156945 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.159848 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.159858 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.159870 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.159897 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.162351 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.163032 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.163037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.163040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.163147 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.163540 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.163543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.163545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.163852 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.164007 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.164345 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.164592 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":453,"thread_start_us":323,"threads_started":1}
I20260811 06:22:36.167016 31736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:36.167953 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.167561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34208)
0811 06:22:36.167721 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.167826 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:22:36.168160 31638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.169533 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.169244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34220)
0811 06:22:36.169377 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.169447 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:22:36.169735 31736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
I20260811 06:22:36.188005 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:45209
--webserver_interface=127.22.121.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:22:36.218988 31736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 1
I20260811 06:22:36.220003 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.219618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34222)
0811 06:22:36.219778 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.219870 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:22:36.220865 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.220565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34234)
0811 06:22:36.220701 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.220772 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 06:22:36.221040 31736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.289234 31736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 2
I20260811 06:22:36.290290 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.289869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34246)
0811 06:22:36.290041 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.290140 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:22:36.291227 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.290921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34260)
0811 06:22:36.291042 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.291116 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 06:22:36.291450 31736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.349032 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } attempt: 5
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:36.361838 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.349970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.22.121.61:39071)
0811 06:22:36.350195 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.350200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.350216 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:22:36.350267 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.350271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.350332 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.350419 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.351243 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.351968 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.353130 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.353566 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.353677 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.356936 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.356940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.356943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.356975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.359419 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.360025 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.360031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.360035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.360153 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.360506 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.360512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.360515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.360796 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.360929 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.361344 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.361544 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:22:36.361838 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.349725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49510)
0811 06:22:36.349919 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.349939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.350058 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.350491 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.350496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.350897 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.351087 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.351096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.352106 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.352112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.352968 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.352980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.353723 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.353729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.353932 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.354479 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.354502 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.356759 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.359540 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.359546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.359550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.359915 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.360259 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.360264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.360268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.360407 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.360912 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.360917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.361059 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.361218 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.361483 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 06:22:36.362957 31605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:36.363932 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.363592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34274)
0811 06:22:36.363740 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.363810 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:22:36.364848 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.364555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34284)
0811 06:22:36.364679 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.364757 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:22:36.365056 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.382628 31736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 3
I20260811 06:22:36.383548 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.383142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34300)
0811 06:22:36.383328 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.383411 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:22:36.384473 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.384093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34304)
0811 06:22:36.384293 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.384365 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 06:22:36.384678 31736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.416241 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 1
I20260811 06:22:36.417240 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.416834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34306)
0811 06:22:36.416996 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.417078 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:22:36.418167 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.417854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34308)
0811 06:22:36.417994 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.418058 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 06:22:36.418362 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.486543 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 2
I20260811 06:22:36.487623 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.487170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34322)
0811 06:22:36.487380 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.487465 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:22:36.488611 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.488288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34328)
0811 06:22:36.488447 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.488515 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 06:22:36.488806 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
W20260811 06:22:36.506791 31750 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:36.507076 31750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:36.507167 31750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:36.507298 31750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:36.514315 31750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:36.514394 31750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:36.514483 31750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:36.514546 31750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:36.514636 31750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:36.523943 31750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38991,127.22.121.61:39071,127.22.121.60:45209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:45209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.121.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:36.525048 31750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:36.526252 31750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 06:22:36.527855 31736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 4
I20260811 06:22:36.528868 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.528444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34338)
0811 06:22:36.528643 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.528724 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:22:36.529868 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.529490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34350)
0811 06:22:36.529643 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.529731 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:22:36.530088 31736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
W20260811 06:22:36.537227 31758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:36.537268 31764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:36.537192 31768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:36.538368 31750 server_base.cc:1034] running on GCE node
I20260811 06:22:36.539041 31750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:36.540295 31750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:36.541504 31750 hybrid_clock.cc:648] HybridClock initialized: now 1786429356541436 us; error 44 us; skew 500 ppm
I20260811 06:22:36.541803 31750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:36.544689 31750 init.cc:377] Logged in from keytab as kudu/127.22.121.60@KRBTEST.COM (short username kudu)
I20260811 06:22:36.545989 31750 webserver.cc:460] Webserver started at http://127.22.121.60:35049/ using document root <none> and password file <none>
I20260811 06:22:36.546582 31750 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:36.546672 31750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:36.546917 31750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:36.549606 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data/instance:
uuid: "ddb22b76bd774d5981c73fa70e7c5aa4"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6b3146c7070c8ae9b09c563919e3571b"
server_key_iv: "49074847f411bc2321acc64c061c2232"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.550300 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal/instance:
uuid: "ddb22b76bd774d5981c73fa70e7c5aa4"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6b3146c7070c8ae9b09c563919e3571b"
server_key_iv: "49074847f411bc2321acc64c061c2232"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.554884 31750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 06:22:36.557971 31781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:36.559125 31750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:22:36.559255 31750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "ddb22b76bd774d5981c73fa70e7c5aa4"
format_stamp: "Formatted at 2026-08-11 06:22:36 on dist-test-slave-gjrz"
server_key: "6b3146c7070c8ae9b09c563919e3571b"
server_key_iv: "49074847f411bc2321acc64c061c2232"
server_key_version: "encryptionkey@0"
I20260811 06:22:36.559463 31750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:36.586328 31750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:36.589919 31750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:36.590116 31750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:36.595019 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 3
I20260811 06:22:36.596139 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.595607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34356)
0811 06:22:36.595837 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.595936 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 06:22:36.597177 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.596736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34370)
0811 06:22:36.596946 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.597038 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 06:22:36.597415 31605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:45209: Network error: Client connection negotiation failed: client connection to 127.22.121.60:45209: connect: Connection refused (error 111)
I20260811 06:22:36.606767 31750 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.60:45209
I20260811 06:22:36.606774 31869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.60:45209 every 8 connection(s)
I20260811 06:22:36.607941 31750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
I20260811 06:22:36.610145 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 31750
I20260811 06:22:36.610313 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429000386903-23012-0/minicluster-data/master-2/wal/instance
I20260811 06:22:36.610726 23012 external_mini_cluster.cc:1428] Setting key 411b6ced2d26a0c39ab67c1333c97d31
I20260811 06:22:36.611380 31872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:36.620354 31872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:36.632023 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.612367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.22.121.60:45209)
0811 06:22:36.613090 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.613101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.613140 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:22:36.614581 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.614610 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.614958 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.615288 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.617015 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.618282 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.620029 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.620663 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.620799 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.625109 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.625117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.625126 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.625163 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.627833 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.628505 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.628514 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.628522 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.628734 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.629073 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.629078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.629097 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.629716 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.629969 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.630828 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.631168 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":325,"threads_started":1}
I20260811 06:22:36.636205 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.622967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.22.121.62:38991)
0811 06:22:36.623150 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.623155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.623829 (+ 674us) server_negotiation.cc:407] Connection header received
0811 06:22:36.624005 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.624010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.624067 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.624149 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.625146 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.625806 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.627214 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.627675 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.627817 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.631100 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.631104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.631107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.631138 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.633648 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.634323 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.634328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.634332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.634480 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.634927 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.634931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.634933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.635181 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.635343 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.635793 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.635950 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:22:36.636420 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.622970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38991 (local address 127.0.0.1:38570)
0811 06:22:36.623711 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.623748 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.623883 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.624215 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.624221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.624771 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.625011 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.625019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.625963 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.625968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.627077 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.627099 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.627890 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.627897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.628042 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.628625 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.628646 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.630920 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.633761 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.633770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.633776 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.634214 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.634609 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.634615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.634619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.634796 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.635322 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.635330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.635517 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.635713 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.636015 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":355,"threads_started":1}
I20260811 06:22:36.637874 31872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:36.650489 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.638485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:39071 (local address 127.0.0.1:49520)
0811 06:22:36.638633 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.638651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.638736 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.639025 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.639029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.639392 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.639514 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.639521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.640736 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.640742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.641737 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.641748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.642638 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.642642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.642771 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.643219 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.643233 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.645275 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.648181 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.648186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.648189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.648661 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.649020 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.649023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.649025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.649159 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.649718 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.649722 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.649846 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.649956 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.650190 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:22:36.651109 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.634419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.22.121.62:38991)
0811 06:22:36.636450 (+ 2031us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.636456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.636472 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:22:36.636522 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.636526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.636587 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.636682 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.638006 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.639027 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.640481 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.640975 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.641093 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.645218 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.645224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.645228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.645258 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.648126 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.649054 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.649059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.649063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.649212 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.649547 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.649552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.649555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.649858 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.650119 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.650417 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.650756 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1948,"thread_start_us":2125,"threads_started":1}
I20260811 06:22:36.651239 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.638536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49520 (local address 127.22.121.61:39071)
0811 06:22:36.638707 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.638713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.638731 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:22:36.638853 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.638858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.638916 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.638990 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.639655 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.640618 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.641873 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.642501 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.642640 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.645441 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.645446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.645449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.645480 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.648057 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.648760 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.648766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.648769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.648923 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.649267 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.649272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.649275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.649613 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.649738 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.650778 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.650973 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:22:36.652285 31872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:36.666673 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.652693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.22.121.61:39071)
0811 06:22:36.652826 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.652831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.653078 (+ 247us) server_negotiation.cc:407] Connection header received
0811 06:22:36.653139 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.653144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.653201 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.653276 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.654362 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.655030 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.656344 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.656753 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.656840 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.661150 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.661155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.661158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.661187 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.664323 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.664895 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.664900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.664904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.665051 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.665411 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.665416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.665419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.665754 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.665942 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.666224 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.666443 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 06:22:36.667337 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.652827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34398)
0811 06:22:36.652958 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.652973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.653049 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.655676 (+ 2627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.655681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.656075 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.656204 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.656212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.657183 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.657187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.658114 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.658126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.659009 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.659013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.659208 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.659697 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.659712 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.661511 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.664950 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.664953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.664956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.665370 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.665738 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.665741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.665743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.665876 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.666536 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.666540 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.666664 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.666792 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.667040 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":22}
I20260811 06:22:36.667932 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.653050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34398 (local address 127.22.121.60:45209)
0811 06:22:36.655412 (+ 2362us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.655418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.655435 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:22:36.655476 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.655481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.655540 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.655630 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.656334 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.657061 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.658253 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.658871 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.658999 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.661906 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.661912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.661916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.661952 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.664825 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.665473 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.665478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.665481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.665611 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.665990 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.665994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.665996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.666301 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.666435 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.667429 (+ 994us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.667638 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":2258}
I20260811 06:22:36.671190 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:36.673237 31872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: Bootstrap starting.
I20260811 06:22:36.677080 31872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:36.674459 31558 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "47deae91ec664255a2d425567027f3d2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38584
I20260811 06:22:36.678323 31872 log.cc:824] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:36.681165 31872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: No bootstrap required, opened a new log
I20260811 06:22:36.691447 31872 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.691812 31872 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb22b76bd774d5981c73fa70e7c5aa4, State: Initialized, Role: FOLLOWER
I20260811 06:22:36.692278 31872 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.693097 31886 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.693398 31886 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:36.694202 31872 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:22:36.702884 31909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:22:36.703006 31909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:22:36.700917 31750 master_runner.cc:186] Error getting master registration for 127.22.121.62:38991: OK, instance_id { permanent_uuid: "47deae91ec664255a2d425567027f3d2" instance_seqno: 1786429355642769 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:22:36.708457 31750 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:22:36.774631 31605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 4
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429355, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:36.790575 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.775350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34410)
0811 06:22:36.775575 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.775605 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.775745 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.776119 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.776124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.776666 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.776894 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.776907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.778313 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.778319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.779879 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.779895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.780896 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.780902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.781115 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.781745 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.781768 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.784414 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.788037 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.788043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.788047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.788459 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.788845 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.788849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.788853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.788980 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.789592 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.789597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.789778 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.789926 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.790190 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:22:36.790632 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.775422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.22.121.60:45209)
0811 06:22:36.775705 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.775714 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.775735 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:22:36.775877 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.775883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.775967 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.776069 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.777040 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.778173 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.780020 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.780744 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.780864 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.784592 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.784598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.784601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.784630 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.787909 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.788608 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.788612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.788614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.788743 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.789082 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.789087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.789097 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.789392 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.789576 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.790053 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.790276 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:22:36.795200 31605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: Bootstrap starting.
I20260811 06:22:36.798473 31605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:36.799476 31605 log.cc:824] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:36.802022 31605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: No bootstrap required, opened a new log
I20260811 06:22:36.806761 31605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.807047 31605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47deae91ec664255a2d425567027f3d2, State: Initialized, Role: FOLLOWER
I20260811 06:22:36.807468 31605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.808269 31913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.808624 31913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:36.809306 31605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:36.809495 31558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:22:36.809751 31558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:36.813412 31558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.817339 31558 leader_election.cc:288] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [CANDIDATE]: Term 1 election: Requested vote from peers f8c81577f5cf47fca482c4a4b38b202e (127.22.121.61:39071), ddb22b76bd774d5981c73fa70e7c5aa4 (127.22.121.60:45209)
I20260811 06:22:36.817946 31829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "47deae91ec664255a2d425567027f3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4"
I20260811 06:22:36.818423 31829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:36.822072 31829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47deae91ec664255a2d425567027f3d2 in term 1.
I20260811 06:22:36.822813 31518 leader_election.cc:302] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47deae91ec664255a2d425567027f3d2, ddb22b76bd774d5981c73fa70e7c5aa4; no voters:
I20260811 06:22:36.823277 31913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:36.829277 31736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } attempt: 5
I20260811 06:22:36.830322 31913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [term 1 LEADER]: Becoming Leader. State: Replica: 47deae91ec664255a2d425567027f3d2, State: Running, Role: LEADER
I20260811 06:22:36.830897 31913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
W20260811 06:22:36.841849 31792 tablet.cc:2369] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:22:36.841949 31932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:22:36.842072 31932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 06:22:36 dist-test-slave-gjrz krb5kdc[31462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:36.843521 31942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47deae91ec664255a2d425567027f3d2. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.843923 31942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:36.825004 31485 master_runner.cc:186] Error getting master registration for 127.22.121.61:39071: OK, instance_id { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" instance_seqno: 1786429356115149 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:22:36.849291 31485 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:22:36.850437 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.829795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:45209 (local address 127.0.0.1:34416)
0811 06:22:36.829985 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:36.830007 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:36.830123 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:36.832360 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:36.832366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:36.832903 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:36.833120 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.833132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.835258 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.835285 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.836798 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:36.836814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.838633 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.838639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:36.838897 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.840807 (+ 1910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:36.840830 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:36.843547 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:36.847504 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.847510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.847514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.848095 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.848565 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:36.848570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:36.848574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:36.848742 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:36.849453 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:36.849459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:36.849619 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 06:22:36.849752 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 06:22:36.850046 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:22:36.851380 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.830201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.22.121.60:45209)
0811 06:22:36.830357 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.830363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.830384 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:22:36.830431 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.830436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.830504 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.830605 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.833312 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.835031 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.837029 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.837784 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.837904 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.843747 (+ 5843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.843753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.843758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.843794 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.847348 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.848251 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.848257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.848261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.848427 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.848873 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.848878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.848881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.849243 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.849388 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.850768 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.851010 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:22:36.856832 31736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: Bootstrap starting.
I20260811 06:22:36.860690 31736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:36.861940 31736 log.cc:824] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:36.861879 31946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:36.869215 31946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 06:22:36.883296 31525 tablet.cc:2369] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:22:36.889045 31736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: No bootstrap required, opened a new log
I20260811 06:22:36.904197 31829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [term 1 FOLLOWER]: Refusing update from remote peer 47deae91ec664255a2d425567027f3d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:22:36.906271 31942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:22:36.902864 31736 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.906659 31736 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c81577f5cf47fca482c4a4b38b202e, State: Initialized, Role: FOLLOWER
I20260811 06:22:36.917558 31736 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } }
I20260811 06:22:36.921025 31886 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47deae91ec664255a2d425567027f3d2. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.921308 31886 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:36.925987 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.880815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.22.121.61:39071)
0811 06:22:36.880972 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.880979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.903048 (+ 22069us) server_negotiation.cc:407] Connection header received
0811 06:22:36.903870 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.903876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.903950 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.904022 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.909449 (+ 5427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.910482 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.914126 (+ 3644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.914756 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.914851 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.919035 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.919042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.919046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.919085 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.922607 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.923539 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.923545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.923548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.923709 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.924210 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.924216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.924219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.924590 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.924811 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.925404 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.925626 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:22:36.927731 31959 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.929171 31959 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:36.932304 31690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "47deae91ec664255a2d425567027f3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f8c81577f5cf47fca482c4a4b38b202e"
I20260811 06:22:36.932859 31688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:36.936275 31688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [term 1 FOLLOWER]: Refusing update from remote peer 47deae91ec664255a2d425567027f3d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:22:36.937638 31913 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:22:36.939626 31690 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47deae91ec664255a2d425567027f3d2 in term 1.
I20260811 06:22:36.958813 31736 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:36.970228 31942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.970628 31942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:36.971805 31946 catalog_manager.cc:1347] Generated new cluster ID: 24fc780d16cf40eea7d231b35d88ff95
I20260811 06:22:36.971895 31946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:36.975432 31957 mvcc.cc:205] Tried to move back new op lower bound from 7317214645862645760 to 7317214645614518272. Current Snapshot: MvccSnapshot[applied={T|T < 7317214645862645760}]
I20260811 06:22:36.984402 31886 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:36.984668 31886 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:36.985436 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.874278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.22.121.62:38991)
0811 06:22:36.874444 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.874450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.896376 (+ 21926us) server_negotiation.cc:407] Connection header received
0811 06:22:36.898491 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.898498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.898595 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.898676 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.906363 (+ 7687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.907626 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.912466 (+ 4840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.913169 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.913266 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.926814 (+ 13548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.926821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.926825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.926865 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.931323 (+ 4458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.960621 (+ 29298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.960628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.960632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.960835 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.983396 (+ 22561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.983403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.983407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.983835 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.984117 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.984753 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.985038 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:22:36.993368 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:36.896877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.22.121.60:45209)
0811 06:22:36.897095 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:36.897101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:36.904670 (+ 7569us) server_negotiation.cc:407] Connection header received
0811 06:22:36.904727 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:36.904731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:36.904826 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:36.904930 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:36.924766 (+ 19836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.925807 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.963298 (+ 37491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:36.964078 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:36.964222 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:36.968890 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:36.968897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:36.968902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:36.968942 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:36.973384 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.987801 (+ 14417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.987807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.987811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.987990 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:36.991378 (+ 3388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:36.991384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:36.991388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:36.991813 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:36.992071 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:36.992667 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 06:22:36.992950 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:22:37.002935 31946 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:37.004707 31946 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:37.009639 31959 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47deae91ec664255a2d425567027f3d2. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:37.009990 31959 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:37.009728 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:37.010155 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:37.011361 31942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47deae91ec664255a2d425567027f3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47deae91ec664255a2d425567027f3d2" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38991 } } peers { permanent_uuid: "f8c81577f5cf47fca482c4a4b38b202e" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 39071 } } peers { permanent_uuid: "ddb22b76bd774d5981c73fa70e7c5aa4" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 45209 } } }
I20260811 06:22:37.011627 31942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:37.029394 31946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47deae91ec664255a2d425567027f3d2: Generated new TSK 0
I20260811 06:22:37.030359 31946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:22:37.052492 31999 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 06:22:37.117841 31612 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.104180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.22.121.62:38991)
0811 06:22:37.104388 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.104393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.114895 (+ 10502us) server_negotiation.cc:407] Connection header received
0811 06:22:37.116018 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.116025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.116119 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.116196 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.117557 (+ 1361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38604: BlockingRecv error: recv got EOF from 127.0.0.1:38604 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 06:22:37.122345 31877 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.115066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.22.121.60:45209)
0811 06:22:37.115223 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.115228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.120429 (+ 5201us) server_negotiation.cc:407] Connection header received
0811 06:22:37.120802 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.120809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.120936 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.121016 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.122074 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34420: BlockingRecv error: recv got EOF from 127.0.0.1:34420 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 06:22:37.124346 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.104082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.22.121.61:39071)
0811 06:22:37.104297 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.104303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.119640 (+ 15337us) server_negotiation.cc:407] Connection header received
0811 06:22:37.122693 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.122700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.122788 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.122862 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.124113 (+ 1251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49554: BlockingRecv error: recv got EOF from 127.0.0.1:49554 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 06:22:37.153271 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.148835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49564 (local address 127.22.121.61:39071)
0811 06:22:37.149072 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.149078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.151815 (+ 2737us) server_negotiation.cc:407] Connection header received
0811 06:22:37.152029 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.152034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.152141 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.152243 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.153026 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49564: BlockingRecv error: recv got EOF from 127.0.0.1:49564 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260811 06:22:37.162339 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.160436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49580 (local address 127.22.121.61:39071)
0811 06:22:37.160593 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.160598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.161089 (+ 491us) server_negotiation.cc:407] Connection header received
0811 06:22:37.161247 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.161251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.161328 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.161428 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.162156 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49580: BlockingRecv error: recv got EOF from 127.0.0.1:49580 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 06:22:37.169723 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.167775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49596 (local address 127.22.121.61:39071)
0811 06:22:37.167916 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.167921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.168309 (+ 388us) server_negotiation.cc:407] Connection header received
0811 06:22:37.168563 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.168569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.168667 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.168770 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.169518 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49596: BlockingRecv error: recv got EOF from 127.0.0.1:49596 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 06:22:37.178246 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.176503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.22.121.61:39071)
0811 06:22:37.176688 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.176693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.177129 (+ 436us) server_negotiation.cc:407] Connection header received
0811 06:22:37.177269 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.177274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.177343 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.177433 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.178060 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49606: BlockingRecv error: recv got EOF from 127.0.0.1:49606 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 06:22:37.191136 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.189314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.22.121.61:39071)
0811 06:22:37.189468 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.189474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.189974 (+ 500us) server_negotiation.cc:407] Connection header received
0811 06:22:37.190126 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.190131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.190200 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.190296 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.190951 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49614: BlockingRecv error: recv got EOF from 127.0.0.1:49614 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 06:22:37.211877 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.210038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.22.121.61:39071)
0811 06:22:37.210215 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.210221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.210574 (+ 353us) server_negotiation.cc:407] Connection header received
0811 06:22:37.210743 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.210749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.210839 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.210950 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.211675 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49620: BlockingRecv error: recv got EOF from 127.0.0.1:49620 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 06:22:37.250762 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.248872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.22.121.61:39071)
0811 06:22:37.249058 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.249063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.249390 (+ 327us) server_negotiation.cc:407] Connection header received
0811 06:22:37.249585 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.249588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.249662 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.249741 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.250515 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49630: BlockingRecv error: recv got EOF from 127.0.0.1:49630 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 06:22:37.326486 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.318656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.22.121.61:39071)
0811 06:22:37.318885 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.318892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.325114 (+ 6222us) server_negotiation.cc:407] Connection header received
0811 06:22:37.325294 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.325300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.325404 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.325546 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.326254 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49632: BlockingRecv error: recv got EOF from 127.0.0.1:49632 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 06:22:37.459378 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.457574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.22.121.61:39071)
0811 06:22:37.457813 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.457819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.458034 (+ 215us) server_negotiation.cc:407] Connection header received
0811 06:22:37.458194 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.458198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.458270 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.458353 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.459058 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49638: BlockingRecv error: recv got EOF from 127.0.0.1:49638 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:22:37.704790 31909 catalog_manager.cc:1259] Loaded cluster ID: 24fc780d16cf40eea7d231b35d88ff95
I20260811 06:22:37.704881 31909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: loading cluster ID for follower catalog manager: success
I20260811 06:22:37.707324 31909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: acquiring CA information for follower catalog manager: success
I20260811 06:22:37.708990 31909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ddb22b76bd774d5981c73fa70e7c5aa4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:22:37.720386 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:37.718659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.22.121.61:39071)
0811 06:22:37.718873 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:37.718880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:37.719042 (+ 162us) server_negotiation.cc:407] Connection header received
0811 06:22:37.719255 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:37.719279 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:37.719365 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:37.719446 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:37.720190 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49648: BlockingRecv error: recv got EOF from 127.0.0.1:49648 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:22:38.057597 31999 catalog_manager.cc:1259] Loaded cluster ID: 24fc780d16cf40eea7d231b35d88ff95
I20260811 06:22:38.057745 31999 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: loading cluster ID for follower catalog manager: success
I20260811 06:22:38.061575 31999 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: acquiring CA information for follower catalog manager: success
I20260811 06:22:38.064015 31999 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8c81577f5cf47fca482c4a4b38b202e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:22:38.248366 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:38.241154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.22.121.61:39071)
0811 06:22:38.241823 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:38.241829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:38.242153 (+ 324us) server_negotiation.cc:407] Connection header received
0811 06:22:38.242368 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:38.242372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:38.242495 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:38.242610 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:38.243141 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.244006 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.246032 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.246475 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.246573 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:38.247141 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:38.247743 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:38.247854 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:38.247969 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 06:22:38.248042 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":325,"threads_started":1}
I20260811 06:22:38.260306 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:38.254290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.22.121.60:45209)
0811 06:22:38.254813 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:38.254817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:38.254833 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:22:38.254919 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:38.254923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:38.255055 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:38.255195 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:38.255633 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.256300 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.258119 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.258494 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.258577 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:38.259148 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:38.259645 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:38.259753 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:38.259941 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 06:22:38.260006 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":332,"threads_started":1}
I20260811 06:22:38.285756 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:38.267351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.22.121.61:39071)
0811 06:22:38.267539 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:38.267545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:38.278987 (+ 11442us) server_negotiation.cc:407] Connection header received
0811 06:22:38.279291 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:38.279297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:38.279382 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:38.279465 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:38.280237 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.281338 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.283587 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.284210 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.284322 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:38.284948 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:38.285080 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:38.285140 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:38.285367 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 06:22:38.285463 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:22:38.286463 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:38.267695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.22.121.62:38991)
0811 06:22:38.277937 (+ 10242us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:38.277944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:38.278321 (+ 377us) server_negotiation.cc:407] Connection header received
0811 06:22:38.278561 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:38.278567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:38.278764 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:38.278923 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:38.279682 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.280860 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.283198 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.283913 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.284029 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:38.284724 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:38.285147 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:38.285252 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:38.286088 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 06:22:38.286183 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10109,"thread_start_us":9988,"threads_started":1}
I20260811 06:22:38.291411 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:38.279137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.22.121.60:45209)
0811 06:22:38.279326 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:38.279332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:38.285923 (+ 6591us) server_negotiation.cc:407] Connection header received
0811 06:22:38.286167 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:38.286172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:38.286247 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:38.286326 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:38.286864 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.287870 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.289777 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:38.290264 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:38.290334 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:38.290756 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:38.290850 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:38.290921 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:38.291050 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:22:38.291126 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:22:38.291851 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31485
I20260811 06:22:38.292946 31599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:38.403486 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31485
I20260811 06:22:38.429983 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31620
I20260811 06:22:38.431069 31726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:38.545063 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31620
I20260811 06:22:38.568702 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 31750
I20260811 06:22:38.569756 31862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:38.674546 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 31750
2026-08-11T06:22:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4944 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:38 dist-test-slave-gjrz krb5kdc[32146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:38 dist-test-slave-gjrz krb5kdc[32146](info): set up 2 sockets
Aug 11 06:22:38 dist-test-slave-gjrz krb5kdc[32146](info): commencing operation
krb5kdc: starting...
W20260811 06:22:39.940893 23012 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.008s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/test-user.keytab.
Aug 11 06:22:40 dist-test-slave-gjrz krb5kdc[32146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.121.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
Entry for principal kudu/127.22.121.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab.
I20260811 06:22:40.206719 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38263
--webserver_interface=127.22.121.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:40.524848 32167 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:40.525143 32167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:40.525250 32167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:40.525352 32167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:40.532270 32167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:40.532390 32167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:40.532482 32167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:40.532694 32167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:40.532833 32167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:40.542331 32167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.62.keytab
--principal=kudu/127.22.121.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.62:38263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.121.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:40.543505 32167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:40.544819 32167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:40.555859 32173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:40.555891 32178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:40.555951 32174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:40.557145 32167 server_base.cc:1034] running on GCE node
I20260811 06:22:40.557837 32167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:40.559134 32167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:40.560353 32167 hybrid_clock.cc:648] HybridClock initialized: now 1786429360560283 us; error 42 us; skew 500 ppm
I20260811 06:22:40.560657 32167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:40 dist-test-slave-gjrz krb5kdc[32146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:40.563642 32167 init.cc:377] Logged in from keytab as kudu/127.22.121.62@KRBTEST.COM (short username kudu)
I20260811 06:22:40.564985 32167 webserver.cc:460] Webserver started at http://127.22.121.62:45143/ using document root <none> and password file <none>
I20260811 06:22:40.565618 32167 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:40.565716 32167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:40.566005 32167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:40.568852 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data/instance:
uuid: "65ca930b3e9242d4a1ac0e091e24a70c"
format_stamp: "Formatted at 2026-08-11 06:22:40 on dist-test-slave-gjrz"
server_key: "520aca680e24c7f4c0b83ee0f5e5e833"
server_key_iv: "e410a0223a7b5841606069f577154e07"
server_key_version: "encryptionkey@0"
I20260811 06:22:40.569595 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance:
uuid: "65ca930b3e9242d4a1ac0e091e24a70c"
format_stamp: "Formatted at 2026-08-11 06:22:40 on dist-test-slave-gjrz"
server_key: "520aca680e24c7f4c0b83ee0f5e5e833"
server_key_iv: "e410a0223a7b5841606069f577154e07"
server_key_version: "encryptionkey@0"
I20260811 06:22:40.574322 32167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:22:40.577513 32198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:40.578701 32167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:22:40.578853 32167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal
uuid: "65ca930b3e9242d4a1ac0e091e24a70c"
format_stamp: "Formatted at 2026-08-11 06:22:40 on dist-test-slave-gjrz"
server_key: "520aca680e24c7f4c0b83ee0f5e5e833"
server_key_iv: "e410a0223a7b5841606069f577154e07"
server_key_version: "encryptionkey@0"
I20260811 06:22:40.579036 32167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:40.611130 32167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:40.614724 32167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:40.614964 32167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:40.632018 32167 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.62:38263
I20260811 06:22:40.632036 32288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.62:38263 every 8 connection(s)
I20260811 06:22:40.633239 32167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:40.636803 32289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:40.637857 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32167
I20260811 06:22:40.638053 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-0/wal/instance
I20260811 06:22:40.638646 23012 external_mini_cluster.cc:1428] Setting key 7820e042240eeddeea9214cadfcfc219
I20260811 06:22:40.645107 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:40 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:40.666752 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.648430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.22.121.62:38263)
0811 06:22:40.649986 (+ 1556us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:40.649994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:40.650046 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:22:40.651536 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:40.651542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:40.651761 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:40.651927 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:40.653172 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:40.654350 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:40.656287 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:40.656825 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:40.656954 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:40.660438 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:40.660445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:40.660456 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:40.660483 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:40.663047 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:40.663748 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:40.663754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:40.663756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:40.663890 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:40.664279 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:40.664282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:40.664284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:40.664718 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:40.664906 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:40.665448 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 06:22:40.665769 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1360,"thread_start_us":1147,"threads_started":1}
I20260811 06:22:40.666829 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.648429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38263 (local address 127.0.0.1:38274)
0811 06:22:40.649226 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.649270 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:40.651170 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:40.651973 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:40.651980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:40.652644 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:40.652957 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:40.652978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:40.654499 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:40.654505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:40.656120 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:40.656135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:40.657079 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:40.657087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:40.657238 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:40.657867 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:40.657899 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:40.660228 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:40.663180 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:40.663191 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:40.663202 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:40.663618 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:40.664001 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:40.664006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:40.664010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:40.664165 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:40.664869 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:40.664877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:40.665065 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 06:22:40.665278 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 06:22:40.665958 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":222,"thread_start_us":313,"threads_started":1}
WARNING: no policy specified for kudu/127.22.121.61@KRBTEST.COM; defaulting to no policy
I20260811 06:22:40.672720 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.61@KRBTEST.COM" created.
I20260811 06:22:40.673830 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.673418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52496)
0811 06:22:40.673590 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.673710 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:22:40.674084 32203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
I20260811 06:22:40.675556 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.675211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52502)
0811 06:22:40.675375 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.675440 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 06:22:40.675753 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:35679: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
Entry for principal kudu/127.22.121.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab.
I20260811 06:22:40.702893 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:35679
--webserver_interface=127.22.121.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:22:40.725063 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } attempt: 1
I20260811 06:22:40.726164 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.725648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52514)
0811 06:22:40.725848 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.726006 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:22:40.727116 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.726737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52530)
0811 06:22:40.726882 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.726972 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:22:40.727332 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:35679: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
I20260811 06:22:40.795588 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } attempt: 2
I20260811 06:22:40.796865 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.796262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52532)
0811 06:22:40.796552 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.796675 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 06:22:40.797956 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.797496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52544)
0811 06:22:40.797667 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.797772 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 06:22:40.798156 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:35679: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
I20260811 06:22:40.889462 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } attempt: 3
I20260811 06:22:40.890731 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.890166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52554)
0811 06:22:40.890414 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.890551 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:22:40.891829 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:40.891393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52564)
0811 06:22:40.891559 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:40.891652 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:22:40.892028 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:35679: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
W20260811 06:22:41.020218 32302 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:41.020499 32302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:41.020569 32302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:41.020645 32302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:41.027516 32302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:41.027595 32302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:41.027657 32302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:41.027705 32302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:41.027763 32302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:41.035322 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } attempt: 4
I20260811 06:22:41.036528 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.035927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52580)
0811 06:22:41.036257 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.036360 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 06:22:41.037374 32302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.61.keytab
--principal=kudu/127.22.121.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.61:35679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.121.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:41.037531 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.037111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52592)
0811 06:22:41.037273 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.037371 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:22:41.037762 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.61:35679: Network error: Client connection negotiation failed: client connection to 127.22.121.61:35679: connect: Connection refused (error 111)
I20260811 06:22:41.038538 32302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:41.039731 32302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:41.050582 32312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:41.050624 32310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:41.050628 32315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:41.051560 32302 server_base.cc:1034] running on GCE node
I20260811 06:22:41.052225 32302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:41.053439 32302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:41.054652 32302 hybrid_clock.cc:648] HybridClock initialized: now 1786429361054595 us; error 29 us; skew 500 ppm
I20260811 06:22:41.054920 32302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:41.059880 32302 init.cc:377] Logged in from keytab as kudu/127.22.121.61@KRBTEST.COM (short username kudu)
I20260811 06:22:41.061378 32302 webserver.cc:460] Webserver started at http://127.22.121.61:40231/ using document root <none> and password file <none>
I20260811 06:22:41.062026 32302 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:41.062117 32302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:41.062414 32302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:41.065295 32302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data/instance:
uuid: "931ff5b8e4574652b8b467da04652c5c"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "a496aee640dbfca411b8d32378dba2bc"
server_key_iv: "4d8b6775e98c7755664c942a67d6c645"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.066058 32302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal/instance:
uuid: "931ff5b8e4574652b8b467da04652c5c"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "a496aee640dbfca411b8d32378dba2bc"
server_key_iv: "4d8b6775e98c7755664c942a67d6c645"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.070858 32302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 06:22:41.074167 32331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:41.075654 32302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:22:41.075901 32302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal
uuid: "931ff5b8e4574652b8b467da04652c5c"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "a496aee640dbfca411b8d32378dba2bc"
server_key_iv: "4d8b6775e98c7755664c942a67d6c645"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.076149 32302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:41.098637 32302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:41.102345 32302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:41.102576 32302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:41.118855 32302 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.61:35679
I20260811 06:22:41.118887 32409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.61:35679 every 8 connection(s)
I20260811 06:22:41.120096 32302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/data/info.pb
I20260811 06:22:41.123605 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32302
I20260811 06:22:41.123629 32410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:41.123770 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-1/wal/instance
I20260811 06:22:41.124279 23012 external_mini_cluster.cc:1428] Setting key 8ebc84cc6af1d68e3b92f90952f18896
I20260811 06:22:41.129904 32410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:41.149603 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.132315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.22.121.62:38263)
0811 06:22:41.132522 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.132528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.133684 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 06:22:41.135475 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.135481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.135561 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.135675 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.137085 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.138079 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.139568 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.140133 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.140268 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.143568 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.143576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.143580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.143623 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.146757 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.147459 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.147465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.147468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.147637 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.148007 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.148012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.148015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.148339 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.148491 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.149103 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.149305 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:22:41.150265 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.132769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38263 (local address 127.0.0.1:38280)
0811 06:22:41.133490 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.133535 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.135178 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.135902 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.135909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.136377 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.136899 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.136915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.138216 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.138219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.139416 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.139434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.140321 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.140325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.140508 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.141108 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.141129 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.143348 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.146888 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.146894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.146905 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.147333 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.147747 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.147750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.147752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.147901 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.148443 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.148449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.148742 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.148943 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.149263 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":288,"threads_started":1}
I20260811 06:22:41.151975 32410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.121.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.121.60@KRBTEST.COM" created.
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:41.166011 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.152747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52600)
0811 06:22:41.152901 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.152916 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.152985 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.154024 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.154029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.154335 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.154459 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.154466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.155850 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.155854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.156775 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.156790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.157789 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.157793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.157942 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.158425 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.158443 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.160259 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.163866 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.163869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.163872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.164209 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.164611 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.164614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.164616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.164734 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.165341 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.165345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.165446 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.165546 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.165756 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:22:41.166306 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.153021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.22.121.61:35679)
0811 06:22:41.153619 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.153624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.153646 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:22:41.153684 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.153689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.153864 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.153966 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.154618 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.155728 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.156902 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.157650 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.157763 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.160461 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.160474 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.160488 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.160519 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.163748 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.164333 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.164340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.164343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.164501 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.164843 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.164847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.164850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.165227 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.165417 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.165715 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.165934 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":348,"threads_started":1}
I20260811 06:22:41.168462 32410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:41.169422 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.168979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34810)
0811 06:22:41.169136 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.169261 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:22:41.169656 32335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.171095 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.170722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34814)
0811 06:22:41.170888 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.170983 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:22:41.171345 32410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
Entry for principal kudu/127.22.121.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab.
I20260811 06:22:41.188853 23012 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
/tmp/dist-test-taskTjGG_n/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:38683
--webserver_interface=127.22.121.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.121.20:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:22:41.220671 32410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 1
I20260811 06:22:41.221802 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.221299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34824)
0811 06:22:41.221514 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.221632 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:22:41.222752 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.222383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34840)
0811 06:22:41.222541 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.222629 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:22:41.222982 32410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.291216 32410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 2
I20260811 06:22:41.292397 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.291862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34844)
0811 06:22:41.292089 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.292206 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:22:41.293406 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.292998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34848)
0811 06:22:41.293172 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.293261 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:22:41.293619 32410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.336990 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } attempt: 5
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:41.351286 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.337752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.22.121.61:35679)
0811 06:22:41.337917 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.337922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.338026 (+ 104us) server_negotiation.cc:407] Connection header received
0811 06:22:41.338241 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.338245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.338324 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.338399 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.339463 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.340152 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.341800 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.342243 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.342332 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.346260 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.346265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.346268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.346299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.348785 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.349540 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.349543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.349546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.349658 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.350034 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.350037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.350039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.350295 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.350400 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.350826 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.350978 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:22:41.351410 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.337658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52614)
0811 06:22:41.337914 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.337939 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.338110 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.338499 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.338504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.339032 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.339330 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.339345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.340296 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.340302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.341645 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.341664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.342408 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.342414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.342727 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.343382 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.343407 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.346070 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.348922 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.348928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.348932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.349423 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.349759 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.349764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.349768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.349942 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.350409 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.350414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.350598 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.350759 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.351031 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 06:22:41.352583 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:41.353510 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.353171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34852)
0811 06:22:41.353317 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.353388 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:22:41.354362 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.354034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34854)
0811 06:22:41.354187 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.354269 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 06:22:41.354543 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.384847 32410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 3
I20260811 06:22:41.385883 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.385384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34862)
0811 06:22:41.385599 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.385701 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 06:22:41.386951 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.386486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34878)
0811 06:22:41.386718 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.386811 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260811 06:22:41.387183 32410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.405736 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 1
I20260811 06:22:41.406651 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.406245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34880)
0811 06:22:41.406417 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.406494 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 06:22:41.407652 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.407358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34894)
0811 06:22:41.407473 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.407541 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 06:22:41.407850 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.476045 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 2
I20260811 06:22:41.477273 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.476772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34904)
0811 06:22:41.477020 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.477115 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:22:41.478416 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.478011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34908)
0811 06:22:41.478210 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.478297 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 06:22:41.478670 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
W20260811 06:22:41.512526 32423 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:22:41.512812 32423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:41.512913 32423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:41.513019 32423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:41.519965 32423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:41.520038 32423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:41.520120 32423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:41.520179 32423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:41.520241 32423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:41.529505 32423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.121.20:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.121.62:38263,127.22.121.61:35679,127.22.121.60:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/krb5kdc/kudu_127.22.121.60.keytab
--principal=kudu/127.22.121.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.121.60:38683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.121.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:41.530438 32410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 4
I20260811 06:22:41.530761 32423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:41.531635 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.531065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34924)
0811 06:22:41.531337 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.531455 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:22:41.532287 32423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 06:22:41.532645 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.532253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34928)
0811 06:22:41.532430 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.532517 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 06:22:41.532877 32410 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
W20260811 06:22:41.543550 32431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:41.543502 32433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:41.543584 32430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:41.544337 32423 server_base.cc:1034] running on GCE node
I20260811 06:22:41.544979 32423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:41.546221 32423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:41.547423 32423 hybrid_clock.cc:648] HybridClock initialized: now 1786429361547358 us; error 32 us; skew 500 ppm
I20260811 06:22:41.547709 32423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:41.550540 32423 init.cc:377] Logged in from keytab as kudu/127.22.121.60@KRBTEST.COM (short username kudu)
I20260811 06:22:41.551987 32423 webserver.cc:460] Webserver started at http://127.22.121.60:40639/ using document root <none> and password file <none>
I20260811 06:22:41.552606 32423 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:41.552699 32423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:41.552994 32423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:41.555788 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data/instance:
uuid: "01a7f178ccb249c390d87c03fa0e0dc9"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "1cc6172afb76def05e20bead8a502281"
server_key_iv: "84ecf023f680045c9df1ebe09ae87f17"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.556532 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal/instance:
uuid: "01a7f178ccb249c390d87c03fa0e0dc9"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "1cc6172afb76def05e20bead8a502281"
server_key_iv: "84ecf023f680045c9df1ebe09ae87f17"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.561318 32423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:22:41.564471 32442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:41.565639 32423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:41.565793 32423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data,/tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal
uuid: "01a7f178ccb249c390d87c03fa0e0dc9"
format_stamp: "Formatted at 2026-08-11 06:22:41 on dist-test-slave-gjrz"
server_key: "1cc6172afb76def05e20bead8a502281"
server_key_iv: "84ecf023f680045c9df1ebe09ae87f17"
server_key_version: "encryptionkey@0"
I20260811 06:22:41.565972 32423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:41.584923 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 3
I20260811 06:22:41.585850 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.585418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34944)
0811 06:22:41.585604 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.585689 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:22:41.586735 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.586394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34958)
0811 06:22:41.586558 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.586632 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 06:22:41.586938 32289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.121.60:38683: Network error: Client connection negotiation failed: client connection to 127.22.121.60:38683: connect: Connection refused (error 111)
I20260811 06:22:41.590660 32423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:41.594235 32423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:41.594462 32423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:41.611763 32423 rpc_server.cc:307] RPC server started. Bound to: 127.22.121.60:38683
I20260811 06:22:41.611784 32545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.121.60:38683 every 8 connection(s)
I20260811 06:22:41.612977 32423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/data/info.pb
I20260811 06:22:41.616310 32548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:41.622045 23012 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu as pid 32423
I20260811 06:22:41.622205 23012 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTjGG_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429000386903-23012-0/minicluster-data/master-2/wal/instance
I20260811 06:22:41.622615 23012 external_mini_cluster.cc:1428] Setting key 36ec3d00d15cf4da740a9487a07a08ab
I20260811 06:22:41.622479 32548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:41.642650 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.625081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.22.121.62:38263)
0811 06:22:41.625287 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.625294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.625968 (+ 674us) server_negotiation.cc:407] Connection header received
0811 06:22:41.627203 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.627207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.627298 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.627383 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.628704 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.629412 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.631120 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.631886 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.632007 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.635996 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.636006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.636011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.636048 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.639444 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.640203 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.640208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.640212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.640386 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.640838 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.640842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.640845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.641176 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.641335 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.641935 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.642304 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:22:41.643322 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.625062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.62:38263 (local address 127.0.0.1:38284)
0811 06:22:41.625807 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.625860 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.626970 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.627470 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.627478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.628186 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.628524 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.628540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.629571 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.629577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.630980 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.630992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.632129 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.632138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.632371 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.633100 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.633127 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.635794 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.639578 (+ 3784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.639590 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.639600 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.640087 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.640515 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.640520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.640524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.640708 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.641330 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.641338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.641545 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.641814 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.642379 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":163,"thread_start_us":318,"threads_started":1}
I20260811 06:22:41.643354 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.624254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.22.121.60:38683)
0811 06:22:41.624961 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.624979 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.625046 (+ 67us) server_negotiation.cc:407] Connection header received
0811 06:22:41.626953 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.626979 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.627302 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.627500 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.628814 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.630247 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.631636 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.632350 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.632459 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.636534 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.636541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.636552 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.636584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.639147 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.639987 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.639995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.639999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.640210 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.640619 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.640625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.640628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.641213 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.641475 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.641817 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.642279 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":510,"thread_start_us":324,"threads_started":1}
I20260811 06:22:41.644709 32548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.62@KRBTEST.COM
I20260811 06:22:41.657037 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.645476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.61:35679 (local address 127.0.0.1:52630)
0811 06:22:41.645634 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.645651 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.645740 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.646065 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.646069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.646468 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.646651 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.646662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.647720 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.647726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.648644 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.648660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.649338 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.649345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.649525 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.650015 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.650031 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.652121 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.654964 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.654968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.654970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.655316 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.655695 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.655699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.655701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.655838 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.656405 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.656409 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.656540 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.656656 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.656825 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:22:41.657524 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.645523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.22.121.61:35679)
0811 06:22:41.645705 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.645715 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.645736 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:22:41.645843 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.645849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.645920 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.646014 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.646798 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.647580 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.648792 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.649200 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.649291 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.652292 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.652296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.652300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.652331 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.654842 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.655430 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.655434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.655437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.655580 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.655947 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.655952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.655956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.656278 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.656418 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.656785 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.656988 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:22:41.658470 32548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:22:41.660622 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.646419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.22.121.62:38263)
0811 06:22:41.646565 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.646569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.646583 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:22:41.646661 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.646664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.646712 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.646802 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.647911 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.648580 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.649921 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.650312 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.650401 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.654918 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.654925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.654930 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.654965 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.657850 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.658571 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.658576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.658579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.658723 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.659235 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.659240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.659244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.659620 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.659841 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.660153 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.660322 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.60@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.121.61@KRBTEST.COM
I20260811 06:22:41.671563 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.659075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34986)
0811 06:22:41.659210 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.659227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.659342 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.659598 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.659601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.660015 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.660175 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.660186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.661251 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.661255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.662296 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.662308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.663096 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.663101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.663252 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.663733 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.663748 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.665829 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.669312 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.669317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.669321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.669685 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.670061 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.670065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.670067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.670188 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.670769 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.670772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.670889 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.671016 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.671200 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 06:22:41.671623 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.659138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.22.121.60:38683)
0811 06:22:41.659319 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.659325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.659341 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:22:41.659418 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.659422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.659484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.659578 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.660321 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.661130 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.662402 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.662946 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.663092 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.666019 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.666025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.666029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.666063 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.669157 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.669803 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.669808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.669812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.669949 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.670302 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.670307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.670310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.670643 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.670790 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.671159 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.671407 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:22:41.676499 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.662132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.22.121.61:35679)
0811 06:22:41.662292 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.662297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.662313 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:22:41.662472 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.662475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.662527 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.662595 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.663920 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.664962 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.666197 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.666648 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.666767 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.671386 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.671392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.671397 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.671432 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.673958 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.674758 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.674763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.674767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.674924 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.675349 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.675352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.675354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.675596 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.675787 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.676119 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.676273 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:22:41.676813 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: Bootstrap starting.
I20260811 06:22:41.680280 32548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:41.680940 23012 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:22:41.681429 32548 log.cc:824] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:41.684245 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: No bootstrap required, opened a new log
I20260811 06:22:41.683511 32241 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "65ca930b3e9242d4a1ac0e091e24a70c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38300
I20260811 06:22:41.691361 32548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.691648 32548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a7f178ccb249c390d87c03fa0e0dc9, State: Initialized, Role: FOLLOWER
I20260811 06:22:41.692060 32548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.692906 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.693285 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.693729 32548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:22:41.702369 32578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:22:41.702494 32578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:22:41.700366 32423 master_runner.cc:186] Error getting master registration for 127.22.121.62:38263: OK, instance_id { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" instance_seqno: 1786429360627243 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:22:41.711238 32423 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:22:41.764140 32289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 4
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429360, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.62@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:41.777846 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.764860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:34998)
0811 06:22:41.765070 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.765092 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.765222 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.765573 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.765578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.765981 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.766142 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.766151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.767185 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.767191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.768195 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.768208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.769044 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.769050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.769249 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.769837 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.769867 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.771979 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.775452 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.775458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.775463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.775832 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.776204 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.776209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.776213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.776354 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.776917 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.776921 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.777077 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.777198 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.777456 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 06:22:41.777868 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.764910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.22.121.60:38683)
0811 06:22:41.765132 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.765138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.765159 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:22:41.765355 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.765361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.765429 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.765514 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.766297 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.767029 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.768343 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.768900 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.769010 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.772163 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.772168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.772171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.772202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.775334 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.775952 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.775958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.775962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.776091 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.776462 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.776467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.776471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.776792 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.776940 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.777318 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.777508 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:22:41.783073 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: Bootstrap starting.
I20260811 06:22:41.786937 32289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:41.788215 32289 log.cc:824] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:41.790824 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: No bootstrap required, opened a new log
I20260811 06:22:41.797992 32289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.798367 32289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65ca930b3e9242d4a1ac0e091e24a70c, State: Initialized, Role: FOLLOWER
I20260811 06:22:41.798810 32289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.799772 32583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.800130 32583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.800770 32289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:41.800904 32241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:22:41.801103 32241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:41.804415 32241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.806000 32241 leader_election.cc:288] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [CANDIDATE]: Term 1 election: Requested vote from peers 931ff5b8e4574652b8b467da04652c5c (127.22.121.61:35679), 01a7f178ccb249c390d87c03fa0e0dc9 (127.22.121.60:38683)
I20260811 06:22:41.806970 32484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "01a7f178ccb249c390d87c03fa0e0dc9"
I20260811 06:22:41.807451 32484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:41.810746 32484 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65ca930b3e9242d4a1ac0e091e24a70c in term 1.
I20260811 06:22:41.811308 32202 leader_election.cc:302] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a7f178ccb249c390d87c03fa0e0dc9, 65ca930b3e9242d4a1ac0e091e24a70c; no voters:
I20260811 06:22:41.811832 32583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:41.815420 32583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [term 1 LEADER]: Becoming Leader. State: Replica: 65ca930b3e9242d4a1ac0e091e24a70c, State: Running, Role: LEADER
I20260811 06:22:41.815941 32583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.819200 32595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ca930b3e9242d4a1ac0e091e24a70c. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.819754 32595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:41.832039 32410 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } attempt: 5
I20260811 06:22:41.839852 32612 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Aug 11 06:22:41 dist-test-slave-gjrz krb5kdc[32146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429361, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.121.61@KRBTEST.COM for kudu/127.22.121.60@KRBTEST.COM
I20260811 06:22:41.844419 32605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 06:22:41.846518 32453 tablet.cc:2369] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:22:41.844378 32167 master_runner.cc:186] Error getting master registration for 127.22.121.61:35679: OK, instance_id { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" instance_seqno: 1786429361114261 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:22:41.855342 32167 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:22:41.855412 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.833526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.22.121.60:38683)
0811 06:22:41.833713 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.833719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.833737 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:22:41.833792 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.833797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.833871 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.833962 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.835204 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.836425 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.838298 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.838999 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.839113 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.848312 (+ 9199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.848319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.848324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.848353 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.851826 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.852921 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.852926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.852930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.853073 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.853645 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.853650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.853654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.853967 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.854092 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.854817 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.855039 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:22:41.856604 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.832682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.121.60:38683 (local address 127.0.0.1:35006)
0811 06:22:41.833069 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:41.833089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:41.833211 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:41.834199 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:41.834204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:41.834750 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:41.834984 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.834996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.836645 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.836652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.838088 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:41.838104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.839352 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.839358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:41.839571 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.840286 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:41.840311 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:41.848140 (+ 7829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:41.851963 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.851969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.851973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.852734 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.853261 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:41.853265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:41.853268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:41.853498 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:41.854228 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:41.854233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:41.854465 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 06:22:41.855587 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0811 06:22:41.856019 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 06:22:41.862043 32410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: Bootstrap starting.
I20260811 06:22:41.865999 32410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:41.867154 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.839470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.22.121.62:38263)
0811 06:22:41.839625 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.839631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.846939 (+ 7308us) server_negotiation.cc:407] Connection header received
0811 06:22:41.847202 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.847207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.847317 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.847397 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.852814 (+ 5417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.854004 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.855952 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.856597 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.856689 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.859937 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.859943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.859948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.859985 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.863339 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.864184 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.864190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.864195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.864372 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.864881 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.864886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.864890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.865222 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.865462 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.866586 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.866806 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:22:41.867345 32410 log.cc:824] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: Log is configured to *not* fsync() on all Append() calls
W20260811 06:22:41.867476 32207 tablet.cc:2369] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:22:41.878320 32410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: No bootstrap required, opened a new log
I20260811 06:22:41.884382 32605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:41.885759 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.856487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.22.121.60:38683)
0811 06:22:41.856634 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.856639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.866272 (+ 9633us) server_negotiation.cc:407] Connection header received
0811 06:22:41.868056 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.868061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.868130 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.868197 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.870594 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.871629 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.873451 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.874104 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.874195 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.877331 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.877337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.877342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.877378 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.880836 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.882118 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.882123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.882127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.882378 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.883127 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.883133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.883136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.883739 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.884106 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.885178 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.885401 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:22:41.885946 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:41.839105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.22.121.61:35679)
0811 06:22:41.839259 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:41.839282 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:41.856055 (+ 16773us) server_negotiation.cc:407] Connection header received
0811 06:22:41.863503 (+ 7448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:41.863508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:41.863583 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:41.863694 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:41.866827 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.867879 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.871285 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:41.871916 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:41.872007 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:41.877722 (+ 5715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:41.877728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:41.877732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:41.877767 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:41.881577 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.882976 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.882981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.882984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.883162 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:41.883924 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:41.883929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:41.883932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:41.884281 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:41.884487 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:41.885420 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 06:22:41.885632 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:22:41.909690 32410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.910148 32410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931ff5b8e4574652b8b467da04652c5c, State: Initialized, Role: FOLLOWER
I20260811 06:22:41.910701 32410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } }
I20260811 06:22:41.911010 32484 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [term 1 FOLLOWER]: Refusing update from remote peer 65ca930b3e9242d4a1ac0e091e24a70c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:22:41.912647 32595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:22:41.917893 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.919955 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.919833 32410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:41.920177 32368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:41.924198 32369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "931ff5b8e4574652b8b467da04652c5c"
I20260811 06:22:41.926568 32368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [term 1 FOLLOWER]: Refusing update from remote peer 65ca930b3e9242d4a1ac0e091e24a70c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:22:41.927428 32595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [LEADER]: Connected to new peer: Peer: permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:22:41.931697 32369 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65ca930b3e9242d4a1ac0e091e24a70c in term 1.
I20260811 06:22:41.962929 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ca930b3e9242d4a1ac0e091e24a70c. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.964435 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.968489 32595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.968772 32595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:41.969794 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.970048 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.970422 32633 mvcc.cc:205] Tried to move back new op lower bound from 7317214666376810496 to 7317214666002608128. Current Snapshot: MvccSnapshot[applied={T|T < 7317214666376810496}]
I20260811 06:22:41.973169 32605 catalog_manager.cc:1347] Generated new cluster ID: ca1be929e0994817b9f3d525e0a46de9
I20260811 06:22:41.973270 32605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:41.974056 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ca930b3e9242d4a1ac0e091e24a70c. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.974354 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:41.976267 32583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.976539 32583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:41.978817 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca930b3e9242d4a1ac0e091e24a70c" member_type: VOTER last_known_addr { host: "127.22.121.62" port: 38263 } } peers { permanent_uuid: "931ff5b8e4574652b8b467da04652c5c" member_type: VOTER last_known_addr { host: "127.22.121.61" port: 35679 } } peers { permanent_uuid: "01a7f178ccb249c390d87c03fa0e0dc9" member_type: VOTER last_known_addr { host: "127.22.121.60" port: 38683 } } }
I20260811 06:22:41.979058 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:22:42.026717 32605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:42.028477 32605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:42.034107 32653 catalog_manager.cc:1259] Loaded cluster ID: ca1be929e0994817b9f3d525e0a46de9
I20260811 06:22:42.034238 32653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: loading cluster ID for follower catalog manager: success
I20260811 06:22:42.037334 32653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: acquiring CA information for follower catalog manager: success
W20260811 06:22:42.038650 32653 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 06:22:42.047036 32605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65ca930b3e9242d4a1ac0e091e24a70c: Generated new TSK 0
I20260811 06:22:42.047912 32605 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:22:42.186373 32295 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.174947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.22.121.62:38263)
0811 06:22:42.175174 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.175180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.183845 (+ 8665us) server_negotiation.cc:407] Connection header received
0811 06:22:42.184823 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.184830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.184918 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.185007 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.186145 (+ 1138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38328: BlockingRecv error: recv got EOF from 127.0.0.1:38328 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:22:42.190742 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.178797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.22.121.61:35679)
0811 06:22:42.178967 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.178973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.187938 (+ 8965us) server_negotiation.cc:407] Connection header received
0811 06:22:42.189506 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.189512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.189619 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.189685 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.190537 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingRecv error: recv got EOF from 127.0.0.1:52672 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 06:22:42.193030 32551 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.188938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.22.121.60:38683)
0811 06:22:42.189128 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.189134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.191340 (+ 2206us) server_negotiation.cc:407] Connection header received
0811 06:22:42.191561 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.191566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.191637 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.191703 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.192799 (+ 1096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35032: BlockingRecv error: recv got EOF from 127.0.0.1:35032 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 06:22:42.231184 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.222303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.22.121.61:35679)
0811 06:22:42.222470 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.222476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.229263 (+ 6787us) server_negotiation.cc:407] Connection header received
0811 06:22:42.229753 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.229759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.229877 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.229946 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.230924 (+ 978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52684: BlockingRecv error: recv got EOF from 127.0.0.1:52684 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:22:42.250547 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.248615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.22.121.61:35679)
0811 06:22:42.248787 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.248793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.249249 (+ 456us) server_negotiation.cc:407] Connection header received
0811 06:22:42.249425 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.249431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.249515 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.249603 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.250325 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52690: BlockingRecv error: recv got EOF from 127.0.0.1:52690 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 06:22:42.266305 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.264390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.22.121.61:35679)
0811 06:22:42.264554 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.264560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.265041 (+ 481us) server_negotiation.cc:407] Connection header received
0811 06:22:42.265209 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.265215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.265284 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.265365 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.266058 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52704: BlockingRecv error: recv got EOF from 127.0.0.1:52704 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 06:22:42.274538 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.272680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.22.121.61:35679)
0811 06:22:42.272907 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.272913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.273366 (+ 453us) server_negotiation.cc:407] Connection header received
0811 06:22:42.273538 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.273542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.273614 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.273694 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.274323 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52712: BlockingRecv error: recv got EOF from 127.0.0.1:52712 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 06:22:42.293104 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.291163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52720 (local address 127.22.121.61:35679)
0811 06:22:42.291375 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.291381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.291884 (+ 503us) server_negotiation.cc:407] Connection header received
0811 06:22:42.292049 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.292055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.292124 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.292205 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.292888 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52720: BlockingRecv error: recv got EOF from 127.0.0.1:52720 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:22:42.313738 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.311757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.22.121.61:35679)
0811 06:22:42.311938 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.311944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.312397 (+ 453us) server_negotiation.cc:407] Connection header received
0811 06:22:42.312595 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.312601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.312674 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.312767 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.313493 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52722: BlockingRecv error: recv got EOF from 127.0.0.1:52722 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:22:42.351228 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.348952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.22.121.61:35679)
0811 06:22:42.349123 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.349129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.349837 (+ 708us) server_negotiation.cc:407] Connection header received
0811 06:22:42.350031 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.350037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.350119 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.350219 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.350958 (+ 739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52732: BlockingRecv error: recv got EOF from 127.0.0.1:52732 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 06:22:42.421190 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.418496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.22.121.61:35679)
0811 06:22:42.418748 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.418755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.419838 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 06:22:42.419982 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.419988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.420103 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.420233 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.420955 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52746: BlockingRecv error: recv got EOF from 127.0.0.1:52746 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 06:22:42.553807 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.552083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.22.121.61:35679)
0811 06:22:42.552249 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.552253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.552680 (+ 427us) server_negotiation.cc:407] Connection header received
0811 06:22:42.552828 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.552832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.552902 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.552980 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.553626 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52750: BlockingRecv error: recv got EOF from 127.0.0.1:52750 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:22:42.704357 32578 catalog_manager.cc:1259] Loaded cluster ID: ca1be929e0994817b9f3d525e0a46de9
I20260811 06:22:42.704457 32578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: loading cluster ID for follower catalog manager: success
I20260811 06:22:42.706885 32578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: acquiring CA information for follower catalog manager: success
I20260811 06:22:42.708696 32578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01a7f178ccb249c390d87c03fa0e0dc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:22:42.815294 32419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:22:42.813513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.22.121.61:35679)
0811 06:22:42.813747 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:42.813753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:42.813941 (+ 188us) server_negotiation.cc:407] Connection header received
0811 06:22:42.814141 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:42.814145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:42.814215 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:42.814305 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:42.815054 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52766: BlockingRecv error: recv got EOF from 127.0.0.1:52766 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:22:43.043905 32653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 931ff5b8e4574652b8b467da04652c5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:22:43.338157 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:43.332555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.22.121.61:35679)
0811 06:22:43.333208 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:43.333213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:43.333232 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:22:43.333299 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:43.333304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:43.333454 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:43.333563 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:43.334011 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.334742 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.336141 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.336554 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.336650 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:43.337148 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:43.337675 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:43.337807 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:43.337845 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 06:22:43.337921 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":444,"threads_started":1}
I20260811 06:22:43.349056 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:43.343971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.22.121.60:38683)
0811 06:22:43.344358 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:43.344362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:43.344427 (+ 65us) server_negotiation.cc:407] Connection header received
0811 06:22:43.344601 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:43.344605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:43.344746 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:43.344919 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:43.345213 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.345859 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.347088 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.347525 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.347619 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:43.348066 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:43.348546 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:43.348666 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:43.348740 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 06:22:43.348800 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":210,"threads_started":1}
I20260811 06:22:43.360967 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:43.354114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.22.121.62:38263)
0811 06:22:43.355483 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:43.355489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:43.355506 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:22:43.355615 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:43.355620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:43.355830 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:43.356037 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:43.356595 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.357414 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.358879 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.359259 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.359407 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:43.359861 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:43.360213 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:43.360313 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:43.360451 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 06:22:43.360530 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":1163,"threads_started":1}
I20260811 06:22:43.362003 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:43.354878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.22.121.60:38683)
0811 06:22:43.355027 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:43.355032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:43.355462 (+ 430us) server_negotiation.cc:407] Connection header received
0811 06:22:43.355623 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:43.355627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:43.355697 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:43.355785 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:43.356545 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.357677 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.359955 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.360498 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.360614 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:43.361246 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:43.361353 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:43.361411 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:43.361676 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 06:22:43.361745 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:22:43.365693 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32167
I20260811 06:22:43.366340 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:43.354136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.22.121.61:35679)
0811 06:22:43.354299 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:43.354304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:43.360833 (+ 6529us) server_negotiation.cc:407] Connection header received
0811 06:22:43.361078 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:43.361084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:43.361170 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:43.361230 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:22:43.362503 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.363241 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.364777 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:43.365175 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:43.365286 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:43.365729 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:22:43.365848 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:22:43.365937 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:43.366042 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 06:22:43.366104 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:22:43.366652 32282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:43.472846 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32167
I20260811 06:22:43.505033 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32302
W20260811 06:22:43.505075 32335 connection.cc:535] server connection from 127.0.0.1:52614 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:22:43.506227 32403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:43.608728 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32302
I20260811 06:22:43.638396 23012 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu pid 32423
I20260811 06:22:43.639467 32537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:43.742607 23012 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTjGG_n/build/asan/bin/kudu with pid 32423
2026-08-11T06:22:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5060 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10005 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (363277 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST